diff --git a/README.md b/README.md index 84f4ad578..ee9c9fa30 100644 --- a/README.md +++ b/README.md @@ -145,6 +145,7 @@ Type `/help` in the panel for full signatures and flags. The most useful ones: | `/schedule [prompt]` | Create a scheduled task | | `/watch [--keep] [--secs <30-120>] [--long \| --short] [/beep]` | Poll the current page and act when a condition is met | | `/workflow` · `/workflow --save ` | Manage saved workflows or compile the last successful run | +| `/teach --start ` · `/teach --end` | Learn a reusable workflow from your demonstrated actions | | `/memory --add ` | Save a user preference | | `/screenshot [--full-page]` | Capture the tab, or the full scrollable page | | `/record [--transcribe]` | Record the current tab, optionally saving a transcript | diff --git a/docs/architecture.md b/docs/architecture.md index 465369000..4ba2cca55 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -531,6 +531,14 @@ action CSS selectors, coordinates, query strings, fragments, and typed values. E value becomes a declared runtime parameter; unsupported or failed actions are skipped and reported to the user as save warnings. +Teacher mode is a second, value-free compiler input for the same schema. +`content/teacher-capture.js` observes only trusted top-document clicks, field +completion, checkbox/radio changes, Enter submissions, and navigation while a +tab-scoped session is active. Field values are never read or sent; a field +action becomes a runtime parameter marker as soon as it crosses the capture +boundary. The sanitized session survives navigation in `storage.session` and +is removed when `/teach --end` compiles and stores the workflow. + Each compiled step contains semantic target metadata (role, accessible name, label, field identity, link, or placeholder), an expected postcondition, and the origin/path family observed before that action. `/workflow` renders a diff --git a/docs/fr/slash-commands.md b/docs/fr/slash-commands.md index 28a0890b1..aebd42ec3 100644 --- a/docs/fr/slash-commands.md +++ b/docs/fr/slash-commands.md @@ -30,6 +30,9 @@ d'une espace ouvre l'autocomplétion de ses options disponibles. | `/workflow --delete ` | Supprimer un workflow enregistré | | `/workflow --export ` | Télécharger un fichier JSON portable `webbrain-workflow/1` assaini | | `/workflow --import --file` | Importer un fichier de workflow portable comme nouveau workflow local | +| `/teach` | Afficher l’état du mode d’apprentissage et le nombre d’actions capturées dans l’onglet actuel | +| `/teach --start ` | Commencer à apprendre un workflow à partir de vos clics et modifications de champs | +| `/teach --end` | Arrêter l’apprentissage et compiler les actions en workflow enregistré sans valeurs saisies | | `/allow-api` | **Dérogation de mutation API par conversation.** Voir [plus bas](#allow-api). | | `/foreground [invite]` | Exécuter une tâche locale au premier plan pour assurer la compatibilité visuelle | | `/dangerously-skip-permissions` | **Contournement global des demandes d'autorisation.** Désactive `Ask before consequential actions` sans ouvrir les Paramètres. WebBrain agira sans demandes par site jusqu'à ce que vous réactiviez le réglage. | diff --git a/docs/privacy-and-data-flow.md b/docs/privacy-and-data-flow.md index 4b955c1b3..575b77482 100644 --- a/docs/privacy-and-data-flow.md +++ b/docs/privacy-and-data-flow.md @@ -135,6 +135,15 @@ postconditions, and parameter descriptors. It does not contain typed field values, raw historical `ref_id` values, action CSS selectors, coordinates, URL query strings, or URL fragments. +`/teach --start ` records a user demonstration in temporary, +tab-scoped session storage. The page capture code never reads field values: +it sends only the field's semantic identity, and the compiler immediately +represents that action as a runtime parameter. Click/field target labels and +sanitized URL origin/path families are retained because replay needs them to +find the same controls. `/teach --end` removes the temporary session whether +compilation succeeds or fails; a successful compilation writes the same +`webbrain-workflow/1` format described above. + `/workflow --run ` collects declared values in a temporary side-panel form and sends them directly to the background replay executor. The values are not written to the workflow, chat text, retry payload, user memory, replay trace, diff --git a/docs/slash-commands.md b/docs/slash-commands.md index a0a8266ff..11dd516bb 100644 --- a/docs/slash-commands.md +++ b/docs/slash-commands.md @@ -30,6 +30,9 @@ for its available flags. | `/workflow --delete ` | Delete a saved workflow | | `/workflow --export ` | Download a sanitized portable `webbrain-workflow/1` JSON file | | `/workflow --import --file` | Import a portable workflow file as a new local saved workflow | +| `/teach` | Show the current tab's Teacher mode status and captured action count | +| `/teach --start ` | Start learning a saved workflow from your demonstrated clicks and field edits | +| `/teach --end` | Stop teaching and compile the captured actions into a value-free saved workflow | | `/allow-api` | **Per-conversation API mutation override.** See [below](#allow-api). | | `/foreground [prompt]` | Run one local task in the foreground for visual compatibility | | `/dangerously-skip-permissions` | **Global permission-prompt bypass.** Turns off `Ask before consequential actions` without opening Settings. WebBrain will act without per-site prompts until you re-enable the setting. | diff --git a/docs/zh-CN/slash-commands.md b/docs/zh-CN/slash-commands.md index d236aa6c9..846200b74 100644 --- a/docs/zh-CN/slash-commands.md +++ b/docs/zh-CN/slash-commands.md @@ -28,6 +28,9 @@ WebBrain 接受作为输入框某行开头的斜杠命令。在面板内输入 ` | `/workflow --delete ` | 删除已保存的工作流 | | `/workflow --export ` | 下载经过清洗的可移植 `webbrain-workflow/1` JSON 文件 | | `/workflow --import --file` | 将可移植工作流文件导入为新的本地工作流 | +| `/teach` | 显示当前标签页的示教状态和已捕获操作数 | +| `/teach --start <名称>` | 开始从您演示的点击和字段编辑中学习工作流 | +| `/teach --end` | 结束示教，并将捕获的操作编译为不含实际输入值的已保存工作流 | | `/allow-api` | **按对话的 API 变更覆盖。** 参见[下文](#allow-api)。 | | `/foreground [提示词]` | 为视觉兼容性在前台运行一次本地任务 | | `/dangerously-skip-permissions` | **全局权限提示绕过。** 无需打开设置即可关闭「操作前询问」。在你重新启用该设置之前，WebBrain 将不再按站点弹出提示。 | diff --git a/src/chrome/manifest.json b/src/chrome/manifest.json index fafae590e..0f2f90ae7 100644 --- a/src/chrome/manifest.json +++ b/src/chrome/manifest.json @@ -48,6 +48,7 @@ "js": [ "src/content/rich-text-toolbar-heuristic.js", "src/content/accessibility-tree.js", + "src/content/teacher-capture.js", "src/content/content.js", "src/content/agent-visual-indicator.js", "src/content/ollama-launch-handoff.js" diff --git a/src/chrome/src/agent/agent.js b/src/chrome/src/agent/agent.js index 929fc0166..593cde79b 100644 --- a/src/chrome/src/agent/agent.js +++ b/src/chrome/src/agent/agent.js @@ -558,6 +558,7 @@ export class Agent extends LoopDetector { this._recentSubmitClicks = new Map(); // tabId -> recent submit click timestamps this._formValidationBlocks = new Map(); // tabId -> validation state that must change before another submit this._runningTabs = new Set(); // tabIds with an active processMessage/Stream in flight + this._runStartGuard = null; // Ordinary runs capture and act in their original tab without activating // it. Managed cloud runs and the explicit /foreground compatibility // override retain the old Page.bringToFront behavior for their lifetime. @@ -896,6 +897,38 @@ export class Agent extends LoopDetector { this._conversationScopeChangeListener = typeof listener === 'function' ? listener : null; } + setRunStartGuard(guard) { + this._runStartGuard = typeof guard === 'function' ? guard : null; + } + + _runEntryKind(defaultKind, runOptions = {}) { + if (runOptions?.cloudRun === true) return 'cloud'; + if (runOptions?.scheduledRun === true) return 'scheduled'; + return defaultKind; + } + + async assertRunStartAllowed(tabId, defaultKind = 'interactive', runOptions = {}) { + await this._runStartGuard?.(tabId, { + kind: this._runEntryKind(defaultKind, runOptions), + runOptions, + }); + } + + async _claimRunEntry(tabId, defaultKind, runOptions = {}) { + if (this._runningTabs.has(tabId)) { + throw new Error('An agent run is already in progress for this tab.'); + } + // Claim synchronously before awaiting the external guard so teacher-mode + // startup cannot race a run whose persisted teacher-state check is pending. + this._runningTabs.add(tabId); + try { + await this.assertRunStartAllowed(tabId, defaultKind, runOptions); + } catch (error) { + this._runningTabs.delete(tabId); + throw error; + } + } + isRunning(tabId) { return this._runningTabs.has(tabId); } @@ -15576,44 +15609,68 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d }; } + async _endSavedWorkflowTraceRun(runId, status, finalContent) { + await trace.endRun(runId, { status, finalContent }); + } + async replaySavedWorkflow(tabId, workflow, parameters = {}, onUpdate = () => {}, runOptions = {}) { - if (this._runningTabs.has(tabId)) throw new Error('An agent run is already in progress for this tab.'); if (!workflow?.id || !Array.isArray(workflow.steps) || !workflow.steps.length) { throw new Error('Saved workflow is missing or invalid.'); } - await this._hydrate(tabId); - // Deterministic workflow replay is an independent task even when it never - // falls back to processMessage. Clear stale selected-text grounding before - // any replay action so a successful replay cannot poison the next turn. - this._clearSelectionGroundingForIndependentRun(tabId, { ...runOptions, independentRun: true }); - // Align pre-run cleanup with processMessage so a prior Act turn cannot - // leak click-AX CDP fallbacks, plan guards, or active-skill state into - // deterministic replay (or the reverse on the next turn). - this._resetActiveSkillsForRun(tabId, { refreshPrompt: false }); - this._clearRunLoopState(tabId); - this._resetRichTextToolbarAudit(tabId); - this._clickAxCdpFallbacks?.delete(tabId); - this.abortFlags.delete(tabId); - this._prepareClarificationAuthorizationForRun(tabId); - this.permissions.beginTurn(tabId); - this.conversationModes.set(tabId, 'act'); - const completionRunToken = this._beginCompletionInvariant(tabId); - this._runningTabs.add(tabId); - const previousForegroundCapture = this._configureCapturePolicyForRun(tabId, runOptions); - const startUrl = await this._currentUrl(tabId); - const conversationId = await this.ensureConversationId(tabId, 'act'); - const traceRunId = await trace.startRun({ - conversationId, - userMessage: `Run saved workflow: ${workflow.name}`, - tabUrl: startUrl, - mode: 'act', - model: this.providerManager?.getActive?.()?.model || '', - providerId: this.providerManager?.activeProviderId || '', - runtimeConfig: this._runtimeTraceConfig(this.providerManager?.getActive?.(), { - tabId, + await this._claimRunEntry(tabId, 'workflow', runOptions); + let completionRunToken = ''; + let previousForegroundCapture = false; + let capturePolicyConfigured = false; + let startUrl = ''; + let traceRunId = null; + try { + await this._hydrate(tabId); + // Deterministic workflow replay is an independent task even when it never + // falls back to processMessage. Clear stale selected-text grounding before + // any replay action so a successful replay cannot poison the next turn. + this._clearSelectionGroundingForIndependentRun(tabId, { ...runOptions, independentRun: true }); + // Align pre-run cleanup with processMessage so a prior Act turn cannot + // leak click-AX CDP fallbacks, plan guards, or active-skill state into + // deterministic replay (or the reverse on the next turn). + this._resetActiveSkillsForRun(tabId, { refreshPrompt: false }); + this._clearRunLoopState(tabId); + this._resetRichTextToolbarAudit(tabId); + this._clickAxCdpFallbacks?.delete(tabId); + this.abortFlags.delete(tabId); + this._prepareClarificationAuthorizationForRun(tabId); + this.permissions.beginTurn(tabId); + this.conversationModes.set(tabId, 'act'); + completionRunToken = this._beginCompletionInvariant(tabId); + previousForegroundCapture = this._configureCapturePolicyForRun(tabId, runOptions); + capturePolicyConfigured = true; + startUrl = await this._currentUrl(tabId); + const conversationId = await this.ensureConversationId(tabId, 'act'); + traceRunId = await trace.startRun({ + conversationId, + userMessage: `Run saved workflow: ${workflow.name}`, + tabUrl: startUrl, mode: 'act', - }), - }); + model: this.providerManager?.getActive?.()?.model || '', + providerId: this.providerManager?.activeProviderId || '', + runtimeConfig: this._runtimeTraceConfig(this.providerManager?.getActive?.(), { + tabId, + mode: 'act', + }), + }); + } catch (error) { + try { + if (capturePolicyConfigured) { + await this._restoreCapturePolicyAfterRun(tabId, previousForegroundCapture); + } + } finally { + try { + if (completionRunToken) this._clearCompletionInvariant(tabId, completionRunToken); + } finally { + this._runningTabs.delete(tabId); + } + } + throw error; + } let traceStatus = 'workflow_stopped'; let finalContent = ''; let matchedSteps = 0; @@ -15859,16 +15916,28 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d healings: verifiedHealings, }; } finally { - await trace.endRun(traceRunId, { status: traceStatus, finalContent }); - this.currentCostState.delete(tabId); - this._planExecutionGuards.delete(tabId); - this._resetActiveSkillsForRun(tabId); - await this._restoreCapturePolicyAfterRun(tabId, previousForegroundCapture); - this._runningTabs.delete(tabId); - this._clearRunLoopState(tabId); - if (traceStatus !== 'workflow_fallback') this._resetRichTextToolbarAudit(tabId); - this._clickAxCdpFallbacks?.delete(tabId); - this._clearCompletionInvariant(tabId, completionRunToken); + try { + try { + await this._endSavedWorkflowTraceRun(traceRunId, traceStatus, finalContent); + } finally { + this.currentCostState.delete(tabId); + this._planExecutionGuards.delete(tabId); + this._resetActiveSkillsForRun(tabId); + this._clearRunLoopState(tabId); + if (traceStatus !== 'workflow_fallback') this._resetRichTextToolbarAudit(tabId); + this._clickAxCdpFallbacks?.delete(tabId); + } + } finally { + try { + await this._restoreCapturePolicyAfterRun(tabId, previousForegroundCapture); + } finally { + try { + this._clearCompletionInvariant(tabId, completionRunToken); + } finally { + this._runningTabs.delete(tabId); + } + } + } } } @@ -21948,10 +22017,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d } async processMessage(tabId, userMessage, onUpdate = () => {}, mode = 'ask', attachments = [], runOptions = {}) { - if (this._runningTabs.has(tabId)) { - throw new Error('An agent run is already in progress for this tab.'); - } - this._runningTabs.add(tabId); + await this._claimRunEntry(tabId, 'interactive', runOptions); try { // Hydration has to run before the toolbar-ledger reset below, so a // persisted obligation cannot outlive the run that cleared it. It is @@ -21972,7 +22038,6 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d if (runOptions?.trustedContinuation !== true) this._continuationExecutionEvidence.delete(tabId); this._clickAxCdpFallbacks.delete(tabId); const completionRunToken = this._beginCompletionInvariant(tabId); - this._runningTabs.add(tabId); if (runOptions?.trustedContinuation !== true) this.persistenceDegradedTabs.delete(tabId); this._runUpdateCallbacks.set(tabId, onUpdate); const previousForegroundCapture = this._configureCapturePolicyForRun(tabId, runOptions); @@ -22874,10 +22939,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d * Process a message with streaming output. */ async processMessageStream(tabId, userMessage, onUpdate = () => {}, mode = 'ask', runOptions = {}) { - if (this._runningTabs.has(tabId)) { - throw new Error('An agent run is already in progress for this tab.'); - } - this._runningTabs.add(tabId); + await this._claimRunEntry(tabId, 'interactive', runOptions); try { // Hydration has to run before the toolbar-ledger reset below, so a // persisted obligation cannot outlive the run that cleared it. It is @@ -22898,7 +22960,6 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d if (runOptions?.trustedContinuation !== true) this._continuationExecutionEvidence.delete(tabId); this._clickAxCdpFallbacks.delete(tabId); const completionRunToken = this._beginCompletionInvariant(tabId); - this._runningTabs.add(tabId); if (runOptions?.trustedContinuation !== true) this.persistenceDegradedTabs.delete(tabId); this._runUpdateCallbacks.set(tabId, onUpdate); const previousForegroundCapture = this._configureCapturePolicyForRun(tabId, runOptions); diff --git a/src/chrome/src/agent/scheduler.js b/src/chrome/src/agent/scheduler.js index 7fb5cd455..7f1b52cc9 100644 --- a/src/chrome/src/agent/scheduler.js +++ b/src/chrome/src/agent/scheduler.js @@ -585,6 +585,10 @@ export class ScheduledJobManager { this.restoreAlarms().catch((e) => console.warn('[WebBrain] restore scheduled alarms failed:', e)); } + isRunning(tabId) { + return this._runningTabs.has(tabId); + } + async _getJobs() { const stored = await this.api.storage.local.get(SCHEDULED_JOBS_KEY); const jobs = stored?.[SCHEDULED_JOBS_KEY]; @@ -1523,20 +1527,49 @@ export class ScheduledJobManager { } let tabId; + let reservedTabId = null; + const releaseReservation = () => { + if (reservedTabId == null) return; + this._runningTabs.delete(reservedTabId); + reservedTabId = null; + }; + const reserveTab = async (candidateTabId) => { + if (candidateTabId == null) return; + const numericTabId = Number(candidateTabId); + const resolvedTabId = Number.isFinite(numericTabId) ? numericTabId : candidateTabId; + if (this._runningTabs.has(resolvedTabId) || this.agent.isRunning(resolvedTabId)) { + throw new Error('An agent run is already in progress for this tab.'); + } + this._runningTabs.add(resolvedTabId); + reservedTabId = resolvedTabId; + try { + await this.agent.assertRunStartAllowed?.(resolvedTabId, 'scheduled', { scheduledRun: true }); + } catch (error) { + releaseReservation(); + throw error; + } + }; try { + const candidateTabId = job.kind === 'resume' || job.target?.type === 'current_tab' + ? (job.tabId || job.target?.tabId) + : job.target?.tabId; + await reserveTab(candidateTabId); tabId = await this._resolveTab(job); + if (reservedTabId !== tabId) { + releaseReservation(); + await reserveTab(tabId); + } await this._validateConversation(job, tabId); await this._validateTaskTarget(job, tabId); } catch (e) { - await this._markFailed(job, e.message); - return; - } - - if (this._runningTabs.has(tabId) || this.agent.isRunning(tabId)) { - await this._requeue(job, 'The target tab already has an active WebBrain run.'); + releaseReservation(); + if (isActiveRunError(e) || e?.code === 'teacher_mode_active') { + await this._requeue(job, 'The target tab already has an active WebBrain run.'); + } else { + await this._markFailed(job, e.message); + } return; } - this._runningTabs.add(tabId); const running = await this._updateJobIf(job.id, (prev) => ( ['pending', 'queued'].includes(prev.status) diff --git a/src/chrome/src/agent/teacher-mode.js b/src/chrome/src/agent/teacher-mode.js new file mode 100644 index 000000000..ea725ea14 --- /dev/null +++ b/src/chrome/src/agent/teacher-mode.js @@ -0,0 +1,226 @@ +import { + normalizeSavedWorkflowName, + normalizeTeacherCaptureAction, + workflowUrlScope, +} from './workflows.js'; + +export const TEACHER_SESSION_PREFIX = 'teacherSession:'; +export const TEACHER_ACTION_LIMIT = 100; +export const TEACHER_RUN_CONFLICT_CODE = 'teacher_mode_active'; + +function teacherRunConflictError() { + const error = new Error('Teacher mode is active for this tab. End teacher mode before starting an automated run.'); + error.code = TEACHER_RUN_CONFLICT_CODE; + return error; +} + +export function createTeacherRunInterlock(store, options = {}) { + const automationOwnsTab = typeof options.automationOwnsTab === 'function' + ? options.automationOwnsTab + : () => false; + let lock = Promise.resolve(); + + const withLock = async (task) => { + const run = lock.then(task, task); + lock = run.catch(() => {}); + return run; + }; + + const blockedResult = async (tabId) => ({ + changed: false, + reason: 'agent_running', + session: await store.get(tabId), + }); + + return { + withLock, + async guardRunStart(tabId) { + return withLock(async () => { + if (await store.get(tabId)) throw teacherRunConflictError(); + }); + }, + async start(tabId, input = {}) { + return withLock(async () => { + if (automationOwnsTab(tabId)) return blockedResult(tabId); + return store.start(tabId, input); + }); + }, + async record(tabId, action) { + return withLock(async () => { + if (automationOwnsTab(tabId)) return blockedResult(tabId); + return store.record(tabId, action); + }); + }, + async navigation(tabId, url, navigationOptions = {}) { + return withLock(async () => { + if (automationOwnsTab(tabId)) return blockedResult(tabId); + return store.navigation(tabId, url, navigationOptions); + }); + }, + }; +} + +function sessionKey(tabId) { + const id = Number(tabId); + return Number.isFinite(id) ? `${TEACHER_SESSION_PREFIX}${id}` : ''; +} + +function normalizedScope(value) { + const origin = String(value?.origin || ''); + const pathFamily = String(value?.pathFamily || '/'); + const scope = workflowUrlScope(`${origin}${pathFamily}`); + return scope?.origin === origin ? scope : null; +} + +function normalizedSession(value, tabId) { + const key = sessionKey(tabId); + const name = normalizeSavedWorkflowName(value?.name); + const start = normalizedScope(value?.start); + const currentScope = normalizedScope(value?.currentScope) || start; + if (!key || value?.active !== true || !name || !start) return null; + const actions = []; + for (const raw of Array.isArray(value?.actions) ? value.actions : []) { + const action = normalizeTeacherCaptureAction(raw); + if (action) actions.push(action); + if (actions.length >= TEACHER_ACTION_LIMIT) break; + } + return { + version: 1, + active: true, + tabId: Number(tabId), + name, + start, + currentScope, + webbrainVersion: String(value.webbrainVersion || '').slice(0, 40), + startedAt: Math.max(1, Number(value.startedAt) || Date.now()), + updatedAt: Math.max(1, Number(value.updatedAt) || Date.now()), + actions, + lastActionAt: Math.max(0, Number(value.lastActionAt) || 0), + lastActionKind: String(value.lastActionKind || ''), + lastActionFingerprint: String(value.lastActionFingerprint || '').slice(0, 2000), + skippedActionCount: Math.max(0, Math.floor(Number(value.skippedActionCount) || 0)), + actionLimitReached: value.actionLimitReached === true, + }; +} + +export function createTeacherSessionStore(storageArea, options = {}) { + const now = options.now || Date.now; + const read = async (tabId) => { + const key = sessionKey(tabId); + if (!key) return null; + const stored = await storageArea.get(key); + return normalizedSession(stored?.[key], tabId); + }; + const write = async (session) => { + const normalized = normalizedSession(session, session?.tabId); + if (!normalized) throw new Error('Invalid teacher session.'); + normalized.updatedAt = now(); + await storageArea.set({ [sessionKey(normalized.tabId)]: normalized }); + return normalized; + }; + + const record = async (tabId, input) => { + const session = await read(tabId); + if (!session) return { changed: false, reason: 'no_active_session', session: null }; + const action = normalizeTeacherCaptureAction(input); + if (!action) { + const reason = ['click', 'field', 'checked'].includes(String(input?.kind || '')) + ? 'unsafe_target' + : 'invalid_action'; + session.skippedActionCount += 1; + return { changed: false, reason, session: await write(session) }; + } + const ts = now(); + const fingerprint = JSON.stringify(action); + if (action.kind === 'field' && action.submit === true && ts - session.lastActionAt < 2000) { + const lastIndex = session.actions.length - 1; + const lastAction = session.actions[lastIndex]; + const candidateIndex = lastAction?.kind === 'navigate' ? lastIndex - 1 : lastIndex; + const candidate = session.actions[candidateIndex]; + if ( + candidate?.kind === 'field' + && candidate.submit !== true + && JSON.stringify({ ...candidate, submit: true }) === fingerprint + ) { + session.actions[candidateIndex] = action; + if (candidateIndex !== lastIndex) session.actions.splice(candidateIndex + 1, 1); + if (candidateIndex === lastIndex) session.currentScope = action.scope; + session.lastActionAt = ts; + session.lastActionKind = action.kind; + session.lastActionFingerprint = fingerprint; + return { changed: true, reason: '', session: await write(session) }; + } + } + if (session.actions.length >= TEACHER_ACTION_LIMIT) { + if (!session.actionLimitReached) { + session.actionLimitReached = true; + await write(session); + } + return { changed: false, reason: 'action_limit', session }; + } + if (fingerprint === session.lastActionFingerprint && ts - session.lastActionAt < 200) { + return { changed: false, reason: 'duplicate', session }; + } + session.actions.push(action); + session.currentScope = action.kind === 'navigate' + ? workflowUrlScope(action.url) || session.currentScope + : action.scope; + session.lastActionAt = ts; + session.lastActionKind = action.kind; + session.lastActionFingerprint = fingerprint; + return { changed: true, reason: '', session: await write(session) }; + }; + + return { + async get(tabId) { return read(tabId); }, + async start(tabId, input = {}) { + const activeSession = await read(tabId); + if (activeSession) return { changed: false, reason: 'already_active', session: activeSession }; + const name = normalizeSavedWorkflowName(input.name); + if (!name) return { changed: false, reason: 'name_required', session: null }; + const start = workflowUrlScope(input.url); + if (!start) return { changed: false, reason: 'http_start_url_required', session: null }; + const ts = now(); + const session = await write({ + version: 1, + active: true, + tabId: Number(tabId), + name, + start, + currentScope: start, + webbrainVersion: input.webbrainVersion, + startedAt: ts, + updatedAt: ts, + actions: [], + }); + return { changed: true, reason: '', session }; + }, + record, + async navigation(tabId, url, options = {}) { + const session = await read(tabId); + if (!session) return { changed: false, reason: 'no_active_session', session: null }; + const destination = workflowUrlScope(url); + if (!destination) return { changed: false, reason: 'invalid_action', session }; + const previous = session.currentScope || session.start; + const sameScope = previous.origin === destination.origin + && previous.pathFamily === destination.pathFamily; + const lastAction = session.actions.at(-1); + const recentNavigationTrigger = options.force !== true + && ( + session.lastActionKind === 'click' + || (session.lastActionKind === 'field' && lastAction?.submit === true) + ) + && now() - session.lastActionAt < 2000; + session.currentScope = destination; + if (sameScope || recentNavigationTrigger) return { changed: true, reason: '', session: await write(session) }; + return record(tabId, { kind: 'navigate', scope: previous, url }); + }, + async clear(tabId) { + const key = sessionKey(tabId); + const session = await read(tabId); + if (!key || !session) return { changed: false, reason: 'no_active_session', session: null }; + await storageArea.remove(key); + return { changed: true, reason: '', session }; + }, + }; +} diff --git a/src/chrome/src/agent/workflows.js b/src/chrome/src/agent/workflows.js index aa323f4b1..775cd8fc2 100644 --- a/src/chrome/src/agent/workflows.js +++ b/src/chrome/src/agent/workflows.js @@ -391,6 +391,135 @@ export function compileWorkflowFromTrace(run, events, options = {}) { : { workflow: null, warnings, reason: 'normalization_failed' }; } +/** Normalize one value-free user-demonstrated action before session storage. */ +export function normalizeTeacherCaptureAction(input) { + const kind = cleanText(input?.kind, 40); + const scope = normalizeWorkflowScope(input?.scope) || workflowUrlScope(input?.pageUrl); + if (!scope) return null; + if (kind === 'navigate') { + const url = safeHttpUrl(input?.url); + return url ? { kind, scope, url } : null; + } + if (!['click', 'field', 'checked'].includes(kind)) return null; + const target = normalizeTarget(input?.target); + if (!isReplayableWorkflowTarget(target)) return null; + if (kind === 'checked' && typeof input?.checked !== 'boolean') return null; + return { + kind, + scope, + target, + ...(kind === 'checked' ? { checked: input.checked } : {}), + ...(kind === 'field' && input?.submit === true ? { submit: true } : {}), + }; +} + +/** Compile a user demonstration into the existing value-free workflow schema. */ +export function compileWorkflowFromDemonstration(input, options = {}) { + const ts = timestamp(options.now, nowMs()); + const name = normalizeSavedWorkflowName(input?.name); + if (!name) return { workflow: null, warnings: [], reason: 'name_required' }; + const start = normalizeWorkflowScope(input?.start) || workflowUrlScope(input?.startUrl); + if (!start) return { workflow: null, warnings: [], reason: 'http_start_url_required' }; + + const rawActions = Array.isArray(input?.actions) ? input.actions : []; + const previouslySkipped = Math.max(0, Math.floor(Number(input?.skippedActionCount) || 0)); + const warnings = previouslySkipped + ? [`Skipped ${previouslySkipped} demonstrated action(s) because no safe, reusable target was available.`] + : []; + if (input?.actionLimitReached === true) { + warnings.push(`Stopped recording additional actions after the ${MAX_STEPS}-step workflow limit.`); + } + const parameters = []; + const steps = []; + let skippedActionCount = previouslySkipped; + for (const raw of rawActions) { + const action = normalizeTeacherCaptureAction(raw); + if (!action) { + skippedActionCount += 1; + warnings.push('Skipped a demonstrated action because it had no safe, reusable target or URL.'); + continue; + } + if (action.kind === 'navigate') { + steps.push({ + id: `step_${steps.length + 1}`, + tool: 'navigate', + args: { url: action.url }, + scope: action.scope, + expected: { kind: 'url_changed' }, + }); + } else if (action.kind === 'click') { + steps.push({ + id: `step_${steps.length + 1}`, + tool: 'click_ax', + args: {}, + target: action.target, + scope: action.scope, + expected: { kind: 'tool_success' }, + }); + } else if (action.kind === 'checked') { + steps.push({ + id: `step_${steps.length + 1}`, + tool: 'set_checked', + args: { checked: action.checked }, + target: action.target, + scope: action.scope, + expected: { kind: 'checked', value: action.checked }, + }); + } else if (parameters.length < MAX_PARAMETERS) { + const base = parameterBase(action.target, `input_${parameters.length + 1}`); + const id = uniqueParameterId(base, parameters); + parameters.push({ + id, + label: cleanText( + action.target.label || action.target.ariaLabel || action.target.name + || action.target.fieldName || id, + 120, + ), + required: true, + sensitive: sensitiveParameter(action.target), + type: 'text', + }); + steps.push({ + id: `step_${steps.length + 1}`, + tool: action.submit ? 'set_field' : 'type_ax', + args: { + text: { [WORKFLOW_PARAM_REF_KEY]: id }, + clear: true, + ...(action.submit ? { submit: true } : {}), + }, + target: action.target, + scope: action.scope, + expected: { kind: 'tool_verified' }, + }); + } else { + skippedActionCount += 1; + warnings.push('Skipped a demonstrated field because the workflow parameter limit was reached.'); + } + if (steps.length >= MAX_STEPS) break; + } + + if (!steps.length) return { workflow: null, warnings, reason: 'no_replayable_steps' }; + const workflow = normalizeSavedWorkflow({ + schema: SAVED_WORKFLOW_SCHEMA, + id: options.id || createSavedWorkflowId(ts), + name, + createdAt: ts, + updatedAt: ts, + source: { runId: '', webbrainVersion: cleanText(input?.webbrainVersion, 40) }, + start, + parameters, + steps, + stats: { + sourceToolCount: rawActions.length + previouslySkipped, + compiledStepCount: steps.length, + skippedToolCount: skippedActionCount, + }, + }, { now: ts }); + return workflow + ? { workflow, warnings, reason: '' } + : { workflow: null, warnings, reason: 'normalization_failed' }; +} + function normalizeParameter(input) { const id = cleanId(input?.id).toLowerCase(); if (!id) return null; diff --git a/src/chrome/src/background.js b/src/chrome/src/background.js index 49de45a3d..25ec124f3 100644 --- a/src/chrome/src/background.js +++ b/src/chrome/src/background.js @@ -14,11 +14,13 @@ import { } from './agent/skills.js'; import { ScheduledJobManager } from './agent/scheduler.js'; import { + compileWorkflowFromDemonstration, compileLatestSuccessfulWorkflow, createSavedWorkflowStore, exportPortableWorkflowDefinition, importPortableWorkflowDefinition, } from './agent/workflows.js'; +import { createTeacherRunInterlock, createTeacherSessionStore } from './agent/teacher-mode.js'; import * as workflowTrace from './trace/recorder.js'; import { startClaudeOAuth, @@ -109,6 +111,14 @@ agent.setConversationScopeChangeListener((tabId, state) => { }); const userMemoryStore = createUserMemoryStore(chrome.storage.local); const savedWorkflowStore = createSavedWorkflowStore(chrome.storage.local); +const teacherSessionStore = createTeacherSessionStore(chrome.storage.session); +const teacherRunInterlock = createTeacherRunInterlock(teacherSessionStore, { + automationOwnsTab: (tabId) => agent.isRunning(tabId) + || detachedRunStarts.has(tabId) + || scheduler.isRunning(tabId) + || cloudRunController.isRunning(tabId), +}); +agent.setRunStartGuard((tabId) => teacherRunInterlock.guardRunStart(tabId)); const profileSync = new ProfileSyncManager(chrome.storage.local); installDownloadDirectoryRouting(chrome); @@ -556,6 +566,33 @@ async function withSavedWorkflowStoreLock(task) { return run; } +async function withTeacherSessionStoreLock(task) { + return teacherRunInterlock.withLock(task); +} + +function publicTeacherSession(session) { + return session ? { + active: true, + name: session.name, + actionCount: session.actions?.length || 0, + startedAt: session.startedAt, + } : { active: false }; +} + +async function notifyTeacherState(tabId, session) { + if (tabId == null) return false; + try { + await chrome.tabs.sendMessage(tabId, { + target: 'content', + action: 'teacher_state', + state: publicTeacherSession(session), + }); + return true; + } catch { + return false; + } +} + async function applyUserMemoryExtractionOperationsToCurrentStore(jobId, operations) { return withUserMemoryStoreLock(async () => { if (!await isUserMemoryExtractionEnabled()) { @@ -1349,6 +1386,7 @@ chrome.tabs.onUpdated.addListener((tabId, changeInfo) => { chrome.tabs.onRemoved.addListener((tabId) => { activeIndicatorTabs.delete(tabId); + withTeacherSessionStoreLock(() => teacherSessionStore.clear(tabId)).catch(() => {}); }); const RUN_UI_PREFIX = 'runUi:'; @@ -1779,9 +1817,18 @@ function recordNav(tabId, type, url) { lastNavByTab.set(tabId, { ts: Date.now(), type, url: url || '' }); } +function recordTeacherNavigation(tabId, url, options) { + teacherRunInterlock.navigation(tabId, url, options).catch(() => {}); +} + +const TEACHER_EXPLICIT_NAVIGATION_TYPES = new Set([ + 'typed', 'auto_bookmark', 'generated', 'keyword', 'keyword_generated', +]); + chrome.webNavigation?.onHistoryStateUpdated?.addListener((details) => { if (details.frameId !== 0) return; recordNav(details.tabId, 'history', details.url); + recordTeacherNavigation(details.tabId, details.url); invalidateContextMenuForTab(details.tabId); }); chrome.webNavigation?.onReferenceFragmentUpdated?.addListener((details) => { @@ -1792,6 +1839,9 @@ chrome.webNavigation?.onReferenceFragmentUpdated?.addListener((details) => { chrome.webNavigation?.onCommitted?.addListener((details) => { if (details.frameId !== 0) return; recordNav(details.tabId, 'committed', details.url); + recordTeacherNavigation(details.tabId, details.url, { + force: TEACHER_EXPLICIT_NAVIGATION_TYPES.has(details.transitionType), + }); invalidateContextMenuForTab(details.tabId); agent.clearDevCssPatchesForTab(details.tabId).catch(() => {}); }); @@ -2176,6 +2226,85 @@ async function handleMessage(msg, sender) { return { ok: true, ...result }; } + // --- Teacher Mode --- + case 'get_teacher_mode': { + const tabId = msg.tabId || sender.tab?.id; + if (!tabId) return { ok: false, reason: 'tab_required', session: { active: false } }; + const session = await teacherSessionStore.get(tabId); + return { ok: true, session: publicTeacherSession(session) }; + } + + case 'start_teacher_mode': { + const tabId = msg.tabId || sender.tab?.id; + if (!tabId) return { ok: false, reason: 'tab_required' }; + if (agent.isRunning(tabId) || detachedRunStarts.has(tabId)) { + return { ok: false, reason: 'agent_running' }; + } + const tab = await chrome.tabs.get(tabId).catch(() => null); + const result = await teacherRunInterlock.start(tabId, { + name: msg.name, + url: tab?.url, + webbrainVersion: chrome.runtime.getManifest().version, + }); + if (result.changed && !await notifyTeacherState(tabId, result.session)) { + await withTeacherSessionStoreLock(() => teacherSessionStore.clear(tabId)); + return { ok: false, reason: 'capture_unavailable', session: { active: false } }; + } + return { ok: result.changed, reason: result.reason, session: publicTeacherSession(result.session) }; + } + + case 'record_teacher_action': { + const tabId = sender.tab?.id; + if (!tabId || (sender.frameId != null && sender.frameId !== 0)) { + return { ok: false, reason: 'tab_required' }; + } + const result = await teacherRunInterlock.record(tabId, msg.teacherAction); + return { + ok: result.changed || ['duplicate', 'unsafe_target'].includes(result.reason), + reason: result.reason, + session: publicTeacherSession(result.session), + }; + } + + case 'end_teacher_mode': { + const tabId = msg.tabId || sender.tab?.id; + if (!tabId) return { ok: false, reason: 'tab_required' }; + const response = await withTeacherSessionStoreLock(async () => { + let session = await teacherSessionStore.get(tabId); + if (!session) return { ok: false, reason: 'no_active_session', stopped: false }; + const flushed = await chrome.tabs.sendMessage(tabId, { + target: 'content', + action: 'flush_teacher_capture', + }).catch(() => null); + if (flushed?.teacherAction) { + await teacherSessionStore.record(tabId, flushed.teacherAction); + session = await teacherSessionStore.get(tabId); + } + let result; + try { + const compiled = compileWorkflowFromDemonstration(session); + if (!compiled.workflow) { + result = { ok: false, ...compiled }; + } else { + const saved = await withSavedWorkflowStoreLock(() => savedWorkflowStore.put(compiled.workflow)); + result = { + ok: saved.changed, + workflow: saved.workflow, + warnings: compiled.warnings, + reason: saved.reason || '', + }; + } + } catch (error) { + result = { ok: false, reason: 'save_failed', error: error?.message || String(error) }; + } finally { + await teacherSessionStore.clear(tabId); + } + return { ...result, stopped: true }; + }); + if (response.stopped) await notifyTeacherState(tabId, null); + return response; + } + // --- Saved Workflows --- case 'list_saved_workflows': return { ok: true, workflows: await savedWorkflowStore.list() }; diff --git a/src/chrome/src/cloud-runs.js b/src/chrome/src/cloud-runs.js index d0c6e5794..f8e55e938 100644 --- a/src/chrome/src/cloud-runs.js +++ b/src/chrome/src/cloud-runs.js @@ -695,6 +695,7 @@ export function createCloudRunController({ } = {}) { const api = chromeApi; const runs = new Map(); + const startingTabs = new Set(); let hydratePromise = null; let persistQueue = Promise.resolve(); let persistTimer = null; @@ -770,7 +771,6 @@ export function createCloudRunController({ if (requestedTabId != null && requestedTabId !== '') { const tab = await api.tabs.get(Number(requestedTabId)); if (!isUsableCloudTab(tab)) throw new Error(`Tab ${requestedTabId} is not a controllable webpage.`); - await activateTab(tab); return tab.id; } @@ -780,10 +780,7 @@ export function createCloudRunController({ const allTabs = await api.tabs.query({}); const fallback = allTabs.find(isUsableCloudTab); - if (fallback) { - await activateTab(fallback); - return fallback.id; - } + if (fallback) return fallback.id; const created = await api.tabs.create({ url: 'about:blank', active: true }); if (created?.id == null) throw new Error('Could not create a browser tab for the cloud run.'); @@ -1030,7 +1027,9 @@ export function createCloudRunController({ ? `Run saved workflow: ${workflow.name}` : String(msg.task || msg.text || '').trim(); if (!task) throw new Error('cloud_run requires `task`.'); - if (agent.isRunning(tabId)) throw new Error(`Tab ${tabId} already has an active WebBrain run.`); + if (startingTabs.has(tabId) || agent.isRunning(tabId)) { + throw new Error(`Tab ${tabId} already has an active WebBrain run.`); + } const apiMutationsAllowed = msg.apiMutationsAllowed === true || msg.api_mutations_allowed === true; if (apiMutationsAllowed && mode !== 'act') { @@ -1050,6 +1049,15 @@ export function createCloudRunController({ if (runs.has(runId)) { throw cloudRunError(`Cloud run ${runId} already exists.`, 409); } + startingTabs.add(tabId); + try { + await agent.assertRunStartAllowed?.(tabId, 'cloud', { cloudRun: true }); + const targetTab = await api.tabs.get(tabId); + await activateTab(targetTab); + } catch (error) { + startingTabs.delete(tabId); + throw error; + } const createdAt = isoNow(); const run = { runId, @@ -1076,7 +1084,13 @@ export function createCloudRunController({ completedAt: null, }; runs.set(run.runId, run); - await persist(); + try { + await persist(); + } catch (error) { + runs.delete(run.runId); + startingTabs.delete(tabId); + throw error; + } (async () => { let recordingId = null; @@ -1191,6 +1205,7 @@ export function createCloudRunController({ ); run.finalUrl = cloudTerminalUrl(redactWorkflowValue(await getTabUrl(tabId)), { strictSecretMode }); } finally { + startingTabs.delete(tabId); // The bridge flag is a run-scoped grant, not the sidebar's persistent // /allow-api setting. Revoke only permission this run added; preserve a // pre-existing per-conversation or global user grant. @@ -1365,6 +1380,7 @@ export function createCloudRunController({ return { runs, + isRunning: (tabId) => startingTabs.has(tabId), startRun, startWorkflowRun, compileWorkflow, diff --git a/src/chrome/src/content/accessibility-tree.js b/src/chrome/src/content/accessibility-tree.js index 968ddf06c..44fd68ff4 100644 --- a/src/chrome/src/content/accessibility-tree.js +++ b/src/chrome/src/content/accessibility-tree.js @@ -1133,5 +1133,6 @@ window.__wb_ax_lookup = lookup; window.__wb_ax_ref = getOrMintRef; window.__wb_ax_name = getAccessibleName; + window.__wb_ax_role = getRole; window.__wb_ax_suggest = suggestNearRefs; })(); diff --git a/src/chrome/src/content/teacher-capture.js b/src/chrome/src/content/teacher-capture.js new file mode 100644 index 000000000..387b0c443 --- /dev/null +++ b/src/chrome/src/content/teacher-capture.js @@ -0,0 +1,251 @@ +/** Captures trusted, value-free user actions while /teach is active. */ +(() => { + if (window.top !== window || window.__webbrainTeacherCaptureInstalled) return; + window.__webbrainTeacherCaptureInstalled = true; + + const INDICATOR_ID = 'webbrain-teacher-indicator'; + const recordedFields = new WeakMap(); + const dirtyFields = new WeakSet(); + let active = false; + let sessionName = ''; + let pendingEnter = null; + + function clean(value, max = 240) { + return String(value || '').replace(/[\r\n]+/g, ' ').replace(/\s+/g, ' ').trim().slice(0, max); + } + + function safeHref(element) { + const raw = element?.getAttribute?.('href'); + if (!raw) return ''; + try { + const url = new URL(raw, location.href); + if (!['http:', 'https:'].includes(url.protocol)) return ''; + url.username = ''; + url.password = ''; + url.search = ''; + url.hash = ''; + return url.toString(); + } catch { + return ''; + } + } + + function inferredRole(element) { + const exported = window.__wb_ax_role?.(element); + if (exported) return clean(exported, 40); + const explicit = clean(element?.getAttribute?.('role'), 40); + if (explicit) return explicit; + const tag = element?.tagName?.toLowerCase?.() || ''; + if (tag === 'button') return 'button'; + if (tag === 'a') return 'link'; + if (tag === 'select') return 'combobox'; + if (tag === 'textarea') return 'textbox'; + if (tag === 'input') { + const type = clean(element.getAttribute('type'), 40).toLowerCase(); + if (type === 'checkbox') return 'checkbox'; + if (type === 'radio') return 'radio'; + if (['button', 'submit'].includes(type)) return 'button'; + return 'textbox'; + } + return 'generic'; + } + + function labelFor(element) { + const labels = Array.from(element?.labels || []) + .map((label) => clean(label.innerText || label.textContent, 120)) + .filter(Boolean); + if (labels.length) return labels.join(' ').slice(0, 120); + return clean(element?.closest?.('label')?.innerText, 120); + } + + function semanticTarget(element, { field = false } = {}) { + if (!(element instanceof Element)) return null; + const target = { + role: inferredRole(element), + name: field ? '' : clean(window.__wb_ax_name?.(element), 120), + label: field && element.matches('[contenteditable]') ? '' : labelFor(element), + id: clean(element.getAttribute('id'), 100), + fieldName: clean(element.getAttribute('name'), 100), + type: clean(element.getAttribute('type'), 40), + ariaLabel: clean(element.getAttribute('aria-label'), 120), + placeholder: clean(element.getAttribute('placeholder'), 120), + href: safeHref(element), + }; + return Object.fromEntries(Object.entries(target).filter(([, item]) => item)); + } + + function isTeacherUiEvent(event) { + return event.composedPath?.().some((node) => node?.id === INDICATOR_ID) === true; + } + + function actionableFromEvent(event) { + const source = event.composedPath?.().find((node) => node instanceof Element); + if (!source) return null; + const direct = source.closest?.([ + 'input', 'textarea', 'select', 'button', 'a[href]', 'label', 'summary', + '[role="button"]', '[role="link"]', '[role="checkbox"]', '[role="radio"]', + '[contenteditable]:not([contenteditable="false"])', '[onclick]', + ].join(',')); + if (direct?.tagName === 'LABEL' && direct.control) return direct.control; + return direct; + } + + function isCheckable(element) { + const type = clean(element?.getAttribute?.('type'), 40).toLowerCase(); + const role = clean(element?.getAttribute?.('role'), 40).toLowerCase(); + return ['checkbox', 'radio'].includes(type) || ['checkbox', 'radio'].includes(role); + } + + function isField(element) { + if (!(element instanceof Element)) return false; + if (element.matches('textarea,select,[contenteditable]:not([contenteditable="false"])')) return true; + if (!element.matches('input')) return false; + const type = clean(element.getAttribute('type') || 'text', 40).toLowerCase(); + return !['button', 'submit', 'reset', 'checkbox', 'radio', 'file', 'hidden', 'image'].includes(type); + } + + function isSubmitControl(element) { + if (!element?.matches?.('button,input')) return false; + const type = clean(element.getAttribute('type'), 40).toLowerCase(); + return element.matches('button') ? (!type || type === 'submit') : ['submit', 'image'].includes(type); + } + + function formFor(element) { + return element?.form || element?.closest?.('form') || null; + } + + function sendAction(action) { + if (!active || !action) return; + try { + chrome.runtime.sendMessage({ + target: 'background', + action: 'record_teacher_action', + teacherAction: { ...action, pageUrl: location.href }, + }).catch?.(() => {}); + } catch { /* extension context may be invalidated during navigation */ } + } + + function activeFieldAction() { + let element = document.activeElement; + while (element?.shadowRoot?.activeElement) element = element.shadowRoot.activeElement; + if (!dirtyFields.has(element) || !isField(element)) return null; + dirtyFields.delete(element); + return { kind: 'field', target: semanticTarget(element, { field: true }), pageUrl: location.href }; + } + + function updateIndicator() { + document.getElementById(INDICATOR_ID)?.remove(); + if (!active || !document.documentElement) return; + const indicator = document.createElement('div'); + indicator.id = INDICATOR_ID; + indicator.setAttribute('role', 'status'); + indicator.setAttribute('aria-live', 'polite'); + indicator.textContent = `● WebBrain Teach${sessionName ? ` · ${sessionName}` : ''}`; + Object.assign(indicator.style, { + position: 'fixed', left: '12px', bottom: '12px', zIndex: '2147483647', + padding: '7px 10px', borderRadius: '999px', background: '#241b3d', + color: '#f7f3ff', border: '1px solid #8b5cf6', font: '600 12px system-ui', + boxShadow: '0 4px 18px rgba(0,0,0,.3)', pointerEvents: 'none', + }); + document.documentElement.appendChild(indicator); + } + + function setState(state) { + active = state?.active === true; + sessionName = active ? clean(state?.name, 80) : ''; + updateIndicator(); + } + + document.addEventListener('click', (event) => { + if (!active || !event.isTrusted || isTeacherUiEvent(event)) return; + const element = actionableFromEvent(event); + if (!element) return; + const keyboardSubmitClick = ( + isSubmitControl(element) + && event.detail === 0 + && pendingEnter?.form + && formFor(element) === pendingEnter.form + && Date.now() - pendingEnter.at < 500 + ); + if (pendingEnter && !keyboardSubmitClick) pendingEnter = null; + if (isField(element) || keyboardSubmitClick) return; + if (isCheckable(element)) { + queueMicrotask(() => { + if (!active) return; + const checked = element.matches('input') + ? element.checked === true + : element.getAttribute('aria-checked') === 'true'; + sendAction({ kind: 'checked', checked, target: semanticTarget(element) }); + }); + return; + } + sendAction({ kind: 'click', target: semanticTarget(element) }); + }, true); + + document.addEventListener('input', (event) => { + if (!active || !event.isTrusted) return; + const element = actionableFromEvent(event); + if (isField(element)) dirtyFields.add(element); + }, true); + + document.addEventListener('change', (event) => { + if (!active || !event.isTrusted) return; + const element = actionableFromEvent(event); + if (!isField(element)) return; + if (Date.now() - (recordedFields.get(element) || 0) < 1000) return; + dirtyFields.delete(element); + sendAction({ kind: 'field', target: semanticTarget(element, { field: true }) }); + }, true); + + document.addEventListener('focusout', (event) => { + if (!active || !event.isTrusted) return; + const element = actionableFromEvent(event); + if (!element?.matches?.('[contenteditable]:not([contenteditable="false"])')) return; + if (!dirtyFields.has(element)) return; + dirtyFields.delete(element); + sendAction({ kind: 'field', target: semanticTarget(element, { field: true }) }); + }, true); + + document.addEventListener('keydown', (event) => { + if (!active || !event.isTrusted) return; + if (event.key !== 'Enter' || event.repeat) { + pendingEnter = null; + return; + } + const element = actionableFromEvent(event); + if (!isField(element) || element.matches('textarea,[contenteditable]')) return; + const action = { kind: 'field', target: semanticTarget(element, { field: true }) }; + const at = Date.now(); + dirtyFields.delete(element); + recordedFields.set(element, at); + pendingEnter = { element, form: formFor(element), action, at }; + sendAction(action); + }, true); + + document.addEventListener('submit', (event) => { + if (!active || !event.isTrusted) return; + const pending = pendingEnter; + if (!pending?.form || event.target !== pending.form || Date.now() - pending.at >= 1000) return; + // Wait until submit propagation finishes so page handlers have a chance to + // cancel autocomplete/custom Enter behavior before we mark it replayable. + queueMicrotask(() => { + if (!active || event.defaultPrevented || pendingEnter !== pending) return; + pendingEnter = null; + recordedFields.set(pending.element, Date.now()); + sendAction({ ...pending.action, submit: true }); + }); + }, true); + + chrome.runtime.onMessage.addListener((message, _sender, sendResponse) => { + if (message?.target !== 'content') return; + if (message.action === 'teacher_state') setState(message.state); + if (message.action === 'flush_teacher_capture') { + sendResponse({ teacherAction: active ? activeFieldAction() : null }); + } + }); + try { + chrome.runtime.sendMessage({ target: 'background', action: 'get_teacher_mode' }, (response) => { + if (!chrome.runtime.lastError) setState(response?.session); + }); + } catch { /* ignore unsupported pages */ } +})(); diff --git a/src/chrome/src/ui/locales/ar.js b/src/chrome/src/ui/locales/ar.js index e26a97824..c03dfc66b 100644 --- a/src/chrome/src/ui/locales/ar.js +++ b/src/chrome/src/ui/locales/ar.js @@ -782,6 +782,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "لم يتم الحفظ: النص فارغ بعد التنظيف أو يبدو أنه يحتوي على سر (كلمة مرور أو مفتاح API أو رمز مميز).", "sp.memory.reason.not_found": "لا توجد ذاكرة محفوظة بهذا المعرّف.", "sp.slash.workflows": "عرض مهام سير العمل المحفوظة وإدارتها", + "sp.slash.teach": "سجّل إجراءاتك كمسار عمل محفوظ", "sp.slash.run_workflow": "تشغيل سير عمل محفوظ حسب المعرّف", "sp.workflows.run_prompt": "تشغيل سير العمل المحفوظ «{name}»", "sp.workflows.parameters_for": "أدخل معلمات «{name}». تُستخدم القيم لهذا التشغيل فقط.", @@ -797,6 +798,13 @@ export default { "sp.workflows.saved": "تم حفظ مهمة سير العمل «{name}».", "sp.workflows.deleted": "تم حذف مهمة سير العمل «{name}».", "sp.workflows.error": "خطأ في مهمة سير العمل: {msg}", + "sp.teach.started": "جارٍ تعليم “{name}”. نفّذ مسار العمل، ثم استخدم /teach --end.", + "sp.teach.status": "جارٍ تعليم “{name}”: تم التقاط {count} من الإجراءات.", + "sp.teach.inactive": "وضع التعليم غير نشط.", + "sp.teach.saved": "تم تعلم مسار العمل “{name}” في {count} من الخطوات.", + "sp.teach.already_active": "وضع التعليم نشط بالفعل في علامة التبويب هذه.", + "sp.teach.agent_running": "أوقف تشغيل الوكيل النشط قبل بدء وضع التعليم.", + "sp.teach.capture_unavailable": "لا يمكن لوضع التعليم التقاط هذه الصفحة. افتح أو أعد تحميل صفحة http(s) عادية وحاول مرة أخرى.", "sp.workflows.reason.no_trace": "لم يتم العثور على تشغيل ناجح ومسجل في هذه المحادثة. فعّل التتبع، وأكمل المهمة، ثم احفظها.", "sp.workflows.reason.no_steps": "لا يحتوي هذا التشغيل على إجراءات يمكن حفظها بأمان لإعادة التشغيل.", "sp.workflows.reason.not_found": "لا توجد مهمة سير عمل محفوظة بهذا المعرّف.", diff --git a/src/chrome/src/ui/locales/bn.js b/src/chrome/src/ui/locales/bn.js index fdaef0814..701959659 100644 --- a/src/chrome/src/ui/locales/bn.js +++ b/src/chrome/src/ui/locales/bn.js @@ -128,6 +128,7 @@ export default { 'sp.slash.show_memory': "সংরক্ষিত ব্যবহারকারী মেমরি দেখান", 'sp.slash.forget_memory': "আইডি দ্বারা একটি সংরক্ষিত মেমরি ভুলে যান", 'sp.slash.workflows': "সংরক্ষিত ওয়ার্কফ্লো তালিকাভুক্ত করুন এবং পরিচালনা করুন", + "sp.slash.teach": "আপনার কাজগুলো সংরক্ষিত ওয়ার্কফ্লো হিসেবে রেকর্ড করুন", 'sp.slash.run_workflow': "আইডি দ্বারা একটি সংরক্ষিত ওয়ার্কফ্লো চালান", 'sp.workflows.run_prompt': "সংরক্ষিত ওয়ার্কফ্লো চালান \"{name}\"", 'sp.workflows.parameters_for': "\"{name}\" এর জন্য পরামিতি লিখুন। মান শুধুমাত্র এই রানের জন্য ব্যবহার করা হয়.", @@ -375,6 +376,13 @@ export default { 'sp.workflows.saved': "সংরক্ষিত ওয়ার্কফ্লো \"{name}\"।", 'sp.workflows.deleted': "\"{name}\" কর্মপ্রবাহ মুছে ফেলা হয়েছে৷", 'sp.workflows.error': "কর্মপ্রবাহ ত্রুটি: {msg}", + "sp.teach.started": "“{name}” শেখানো হচ্ছে। ওয়ার্কফ্লোটি সম্পন্ন করে /teach --end ব্যবহার করুন।", + "sp.teach.status": "“{name}” শেখানো হচ্ছে: {count}টি কাজ ধরা হয়েছে।", + "sp.teach.inactive": "টিচার মোড সক্রিয় নয়।", + "sp.teach.saved": "“{name}” ওয়ার্কফ্লোটি {count}টি ধাপে শেখা হয়েছে।", + "sp.teach.already_active": "এই ট্যাবে টিচার মোড ইতিমধ্যে সক্রিয়।", + "sp.teach.agent_running": "টিচার মোড চালুর আগে সক্রিয় এজেন্ট রানটি বন্ধ করুন।", + "sp.teach.capture_unavailable": "টিচার মোড এই পৃষ্ঠাটি ধরতে পারছে না। একটি সাধারণ http(s) পৃষ্ঠা খুলুন বা রিলোড করে আবার চেষ্টা করুন।", 'sp.workflows.reason.no_trace': "এই কথোপকথনে কোনো সফল রেকর্ড করা রান পাওয়া যায়নি। ট্রেসিং সক্ষম করুন, কাজটি সম্পূর্ণ করুন, তারপরে এটি সংরক্ষণ করুন৷", 'sp.workflows.reason.no_steps': "সেই রানের কোনো অ্যাকশন নেই যা রিপ্লে করার জন্য নিরাপদে সংরক্ষণ করা যায়।", 'sp.workflows.reason.not_found': "কোনও সংরক্ষিত ওয়ার্কফ্লোতে সেই আইডি নেই।", diff --git a/src/chrome/src/ui/locales/de.js b/src/chrome/src/ui/locales/de.js index 63e447c0b..010fbce8c 100644 --- a/src/chrome/src/ui/locales/de.js +++ b/src/chrome/src/ui/locales/de.js @@ -124,6 +124,7 @@ export default { 'sp.slash.show_memory': 'Gespeicherte Benutzermemorie anzeigen', 'sp.slash.forget_memory': 'Gespeicherte Memorie nach ID vergessen', 'sp.slash.workflows': 'Gespeicherte Workflows auflisten und verwalten', + "sp.slash.teach": "Aktionen als gespeicherten Workflow aufzeichnen", 'sp.slash.run_workflow': 'Gespeicherten Workflow nach ID ausführen', 'sp.slash.save_workflow': 'Letzten erfolgreichen aufgezeichneten Durchlauf speichern', 'sp.slash.delete_workflow': 'Gespeicherten Workflow nach ID löschen', @@ -365,6 +366,13 @@ export default { 'sp.workflows.saved': 'Workflow „{name}" gespeichert.', 'sp.workflows.deleted': 'Workflow „{name}" gelöscht.', 'sp.workflows.error': 'Workflow-Fehler: {msg}', + "sp.teach.started": "„{name}“ wird gelernt. Führe den Workflow aus und verwende dann /teach --end.", + "sp.teach.status": "„{name}“ wird gelernt: {count} Aktionen erfasst.", + "sp.teach.inactive": "Der Lernmodus ist nicht aktiv.", + "sp.teach.saved": "Workflow „{name}“ mit {count} Schritten gelernt.", + "sp.teach.already_active": "Der Lernmodus ist in diesem Tab bereits aktiv.", + "sp.teach.agent_running": "Beende den aktiven Agent-Lauf, bevor du den Lernmodus startest.", + "sp.teach.capture_unavailable": "Der Lernmodus kann diese Seite nicht erfassen. Öffne oder lade eine normale http(s)-Seite neu und versuche es erneut.", 'sp.workflows.reason.no_trace': 'In diesem Gespräch wurde kein erfolgreicher aufgezeichneter Durchlauf gefunden. Aktivieren Sie die Verfolgung, schließen Sie die Aufgabe ab und speichern Sie sie dann.', 'sp.workflows.reason.no_steps': 'Dieser Durchlauf enthält keine Aktionen, die sicher für die Wiedergabe gespeichert werden können.', 'sp.workflows.reason.not_found': 'Kein gespeicherter Workflow mit dieser ID.', diff --git a/src/chrome/src/ui/locales/en.js b/src/chrome/src/ui/locales/en.js index 177f21346..43cd20045 100644 --- a/src/chrome/src/ui/locales/en.js +++ b/src/chrome/src/ui/locales/en.js @@ -128,6 +128,7 @@ export default { 'sp.slash.show_memory': 'Show saved user memory', 'sp.slash.forget_memory': 'Forget a saved memory by ID', 'sp.slash.workflows': 'List and manage saved workflows', + "sp.slash.teach": "Record your actions as a saved workflow", 'sp.slash.run_workflow': 'Run a saved workflow by ID', 'sp.workflows.run_prompt': 'Run saved workflow “{name}”', 'sp.workflows.parameters_for': 'Enter parameters for “{name}”. Values are used only for this run.', @@ -375,6 +376,13 @@ export default { 'sp.workflows.saved': 'Saved workflow “{name}”.', 'sp.workflows.deleted': 'Deleted workflow “{name}”.', 'sp.workflows.error': 'Workflow error: {msg}', + "sp.teach.started": "Teaching “{name}”. Perform the workflow, then use /teach --end.", + "sp.teach.status": "Teaching “{name}”: {count} actions captured.", + "sp.teach.inactive": "Teacher mode is not active.", + "sp.teach.saved": "Learned workflow “{name}” with {count} steps.", + "sp.teach.already_active": "Teacher mode is already active in this tab.", + "sp.teach.agent_running": "Stop the active agent run before starting Teacher mode.", + "sp.teach.capture_unavailable": "Teacher mode cannot capture this page. Open or reload a regular http(s) page and try again.", 'sp.workflows.reason.no_trace': 'No successful recorded run was found in this conversation. Enable tracing, complete the task, then save it.', 'sp.workflows.reason.no_steps': 'That run has no actions that can be safely saved for replay.', 'sp.workflows.reason.not_found': 'No saved workflow has that ID.', diff --git a/src/chrome/src/ui/locales/es.js b/src/chrome/src/ui/locales/es.js index 49a1ebe22..038a09466 100644 --- a/src/chrome/src/ui/locales/es.js +++ b/src/chrome/src/ui/locales/es.js @@ -782,6 +782,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "No se guardó: el texto queda vacío tras la limpieza o parece contener un secreto (contraseña, clave de API o token).", "sp.memory.reason.not_found": "Ninguna memoria guardada tiene ese ID.", "sp.slash.workflows": "Listar y administrar flujos de trabajo guardados", + "sp.slash.teach": "Graba tus acciones como un flujo de trabajo guardado", "sp.slash.run_workflow": "Ejecutar un flujo guardado por ID", "sp.workflows.run_prompt": "Ejecutar el flujo guardado «{name}»", "sp.workflows.parameters_for": "Introduce los parámetros de «{name}». Los valores solo se usan en esta ejecución.", @@ -797,6 +798,13 @@ export default { "sp.workflows.saved": "Flujo de trabajo «{name}» guardado.", "sp.workflows.deleted": "Flujo de trabajo «{name}» eliminado.", "sp.workflows.error": "Error del flujo de trabajo: {msg}", + "sp.teach.started": "Enseñando “{name}”. Completa el flujo y usa /teach --end.", + "sp.teach.status": "Enseñando “{name}”: {count} acciones capturadas.", + "sp.teach.inactive": "El modo de enseñanza no está activo.", + "sp.teach.saved": "Se aprendió el flujo “{name}” con {count} pasos.", + "sp.teach.already_active": "El modo de enseñanza ya está activo en esta pestaña.", + "sp.teach.agent_running": "Detén la ejecución activa del agente antes de iniciar el modo de enseñanza.", + "sp.teach.capture_unavailable": "El modo de enseñanza no puede capturar esta página. Abre o recarga una página http(s) normal e inténtalo de nuevo.", "sp.workflows.reason.no_trace": "No se encontró una ejecución registrada y correcta en esta conversación. Activa el seguimiento, completa la tarea y guárdala.", "sp.workflows.reason.no_steps": "Esa ejecución no contiene acciones que puedan guardarse de forma segura.", "sp.workflows.reason.not_found": "No existe un flujo de trabajo guardado con ese ID.", diff --git a/src/chrome/src/ui/locales/fa.js b/src/chrome/src/ui/locales/fa.js index ce3974d0f..4f819d1f4 100644 --- a/src/chrome/src/ui/locales/fa.js +++ b/src/chrome/src/ui/locales/fa.js @@ -128,6 +128,7 @@ export default { 'sp.slash.show_memory': "نمایش حافظه کاربر ذخیره شده", 'sp.slash.forget_memory': "یک حافظه ذخیره شده توسط ID را فراموش کنید", 'sp.slash.workflows': "گردش کار ذخیره شده را فهرست و مدیریت کنید", + "sp.slash.teach": "اقدامات خود را به‌صورت گردش‌کار ذخیره‌شده ضبط کنید", 'sp.slash.run_workflow': "یک گردش کار ذخیره شده توسط ID را اجرا کنید", 'sp.workflows.run_prompt': "اجرای گردش کار ذخیره شده \"{name}\"", 'sp.workflows.parameters_for': "پارامترهای \"{name}\" را وارد کنید. مقادیر فقط برای این اجرا استفاده می شود.", @@ -375,6 +376,13 @@ export default { 'sp.workflows.saved': "گردش کار ذخیره شده \"{name}\".", 'sp.workflows.deleted': "گردش کار حذف شده \"{name}\".", 'sp.workflows.error': "خطای گردش کار: {msg}", + "sp.teach.started": "در حال آموزش «{name}». گردش‌کار را انجام دهید، سپس /teach --end را وارد کنید.", + "sp.teach.status": "در حال آموزش «{name}»: {count} اقدام ثبت شد.", + "sp.teach.inactive": "حالت آموزش فعال نیست.", + "sp.teach.saved": "گردش‌کار «{name}» با {count} مرحله آموخته شد.", + "sp.teach.already_active": "حالت آموزش از قبل در این برگه فعال است.", + "sp.teach.agent_running": "پیش از شروع حالت آموزش، اجرای فعال عامل را متوقف کنید.", + "sp.teach.capture_unavailable": "حالت آموزش نمی‌تواند این صفحه را ثبت کند. یک صفحه معمولی http(s) را باز یا بازخوانی کنید و دوباره تلاش کنید.", 'sp.workflows.reason.no_trace': "هیچ اجرا ضبط شده موفقی در این مکالمه یافت نشد. ردیابی را فعال کنید، کار را کامل کنید، سپس آن را ذخیره کنید.", 'sp.workflows.reason.no_steps': "این اجرا هیچ عملکردی ندارد که بتوان با خیال راحت برای پخش مجدد ذخیره کرد.", 'sp.workflows.reason.not_found': "هیچ گردش کار ذخیره شده ای آن شناسه را ندارد.", diff --git a/src/chrome/src/ui/locales/fr.js b/src/chrome/src/ui/locales/fr.js index 2905fa233..0cfb8a663 100644 --- a/src/chrome/src/ui/locales/fr.js +++ b/src/chrome/src/ui/locales/fr.js @@ -782,6 +782,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "Non enregistré : le texte est vide après nettoyage ou semble contenir un secret (mot de passe, clé API, jeton).", "sp.memory.reason.not_found": "Aucune mémoire enregistrée ne possède cet identifiant.", "sp.slash.workflows": "Lister et gérer les workflows enregistrés", + "sp.slash.teach": "Enregistrer vos actions comme workflow sauvegardé", "sp.slash.run_workflow": "Exécuter un workflow enregistré par ID", "sp.workflows.run_prompt": "Exécuter le workflow enregistré « {name} »", "sp.workflows.parameters_for": "Saisissez les paramètres de « {name} ». Les valeurs ne servent qu’à cette exécution.", @@ -797,6 +798,13 @@ export default { "sp.workflows.saved": "Workflow « {name} » enregistré.", "sp.workflows.deleted": "Workflow « {name} » supprimé.", "sp.workflows.error": "Erreur de workflow : {msg}", + "sp.teach.started": "Apprentissage de « {name} ». Effectuez le workflow, puis utilisez /teach --end.", + "sp.teach.status": "Apprentissage de « {name} » : {count} actions capturées.", + "sp.teach.inactive": "Le mode d’apprentissage n’est pas actif.", + "sp.teach.saved": "Workflow « {name} » appris avec {count} étapes.", + "sp.teach.already_active": "Le mode d’apprentissage est déjà actif dans cet onglet.", + "sp.teach.agent_running": "Arrêtez l’exécution active de l’agent avant de lancer le mode d’apprentissage.", + "sp.teach.capture_unavailable": "Le mode d’apprentissage ne peut pas capturer cette page. Ouvrez ou rechargez une page http(s) normale, puis réessayez.", "sp.workflows.reason.no_trace": "Aucune exécution réussie et tracée dans cette conversation. Activez le traçage, terminez la tâche, puis enregistrez-la.", "sp.workflows.reason.no_steps": "Cette exécution ne contient aucune action pouvant être enregistrée de manière sûre.", "sp.workflows.reason.not_found": "Aucun workflow enregistré ne possède cet ID.", diff --git a/src/chrome/src/ui/locales/he.js b/src/chrome/src/ui/locales/he.js index 805df3b0e..4dfc0a16c 100644 --- a/src/chrome/src/ui/locales/he.js +++ b/src/chrome/src/ui/locales/he.js @@ -423,6 +423,7 @@ export default { "st.tab.vision": "ראייה", "st.tab.multimodal": "מולטי-מודאלי", "sp.slash.workflows": "הצגה וניהול של תהליכי עבודה שמורים", + "sp.slash.teach": "הקלטת הפעולות שלך כתהליך עבודה שמור", "sp.slash.run_workflow": "הפעלת תהליך עבודה שמור לפי מזהה", "sp.workflows.run_prompt": "הפעלת תהליך העבודה השמור „{name}”", "sp.workflows.parameters_for": "יש להזין פרמטרים עבור „{name}”. הערכים משמשים להרצה זו בלבד.", @@ -438,6 +439,13 @@ export default { "sp.workflows.saved": "תהליך העבודה „{name}” נשמר.", "sp.workflows.deleted": "תהליך העבודה „{name}” נמחק.", "sp.workflows.error": "שגיאת תהליך עבודה: {msg}", + "sp.teach.started": "מלמד את “{name}”. בצע את התהליך ואז השתמש ב־/teach --end.", + "sp.teach.status": "מלמד את “{name}”: נקלטו {count} פעולות.", + "sp.teach.inactive": "מצב לימוד אינו פעיל.", + "sp.teach.saved": "תהליך העבודה “{name}” נלמד עם {count} שלבים.", + "sp.teach.already_active": "מצב לימוד כבר פעיל בכרטיסייה זו.", + "sp.teach.agent_running": "יש לעצור את הרצת הסוכן הפעילה לפני הפעלת מצב לימוד.", + "sp.teach.capture_unavailable": "מצב לימוד אינו יכול לקלוט דף זה. פתח או טען מחדש דף http(s) רגיל ונסה שוב.", "sp.workflows.reason.no_trace": "לא נמצאה בשיחה זו הרצה מוצלחת ומוקלטת. הפעילו מעקב, השלימו את המשימה ושמרו אותה.", "sp.workflows.reason.no_steps": "בהרצה זו אין פעולות שניתן לשמור בבטחה להפעלה חוזרת.", "sp.workflows.reason.not_found": "אין תהליך עבודה שמור עם מזהה זה.", diff --git a/src/chrome/src/ui/locales/hi.js b/src/chrome/src/ui/locales/hi.js index a18fa4a8e..cf6e74857 100644 --- a/src/chrome/src/ui/locales/hi.js +++ b/src/chrome/src/ui/locales/hi.js @@ -128,6 +128,7 @@ export default { 'sp.slash.show_memory': "सहेजी गई उपयोगकर्ता मेमोरी दिखाएँ", 'sp.slash.forget_memory': "आईडी द्वारा सहेजी गई मेमोरी को भूल जाएं", 'sp.slash.workflows': "सहेजे गए वर्कफ़्लो को सूचीबद्ध करें और प्रबंधित करें", + "sp.slash.teach": "अपने कार्यों को सहेजे गए वर्कफ़्लो के रूप में रिकॉर्ड करें", 'sp.slash.run_workflow': "आईडी द्वारा सहेजा गया वर्कफ़्लो चलाएँ", 'sp.workflows.run_prompt': "सहेजे गए वर्कफ़्लो \"{name}\" चलाएँ", 'sp.workflows.parameters_for': "\"{name}\" के लिए पैरामीटर दर्ज करें। मानों का उपयोग केवल इस रन के लिए किया जाता है।", @@ -375,6 +376,13 @@ export default { 'sp.workflows.saved': "सहेजा गया वर्कफ़्लो \"{name}\"।", 'sp.workflows.deleted': "हटाया गया वर्कफ़्लो \"{name}\"।", 'sp.workflows.error': "वर्कफ़्लो त्रुटि: {msg}", + "sp.teach.started": "“{name}” सिखाया जा रहा है। वर्कफ़्लो पूरा करें, फिर /teach --end चलाएँ।", + "sp.teach.status": "“{name}” सिखाया जा रहा है: {count} कार्य कैप्चर हुए।", + "sp.teach.inactive": "टीचर मोड सक्रिय नहीं है।", + "sp.teach.saved": "“{name}” वर्कफ़्लो {count} चरणों के साथ सीख लिया गया।", + "sp.teach.already_active": "इस टैब में टीचर मोड पहले से सक्रिय है।", + "sp.teach.agent_running": "टीचर मोड शुरू करने से पहले सक्रिय एजेंट रन रोकें।", + "sp.teach.capture_unavailable": "टीचर मोड इस पेज को कैप्चर नहीं कर सकता। किसी सामान्य http(s) पेज को खोलें या रीलोड करें और फिर कोशिश करें।", 'sp.workflows.reason.no_trace': "इस बातचीत में कोई सफल रिकॉर्डेड रन नहीं मिला. ट्रेसिंग सक्षम करें, कार्य पूरा करें, फिर इसे सहेजें।", 'sp.workflows.reason.no_steps': "उस रन में ऐसी कोई क्रिया नहीं है जिसे पुनः चलाने के लिए सुरक्षित रूप से सहेजा जा सके।", 'sp.workflows.reason.not_found': "किसी भी सहेजे गए वर्कफ़्लो में वह आईडी नहीं है।", diff --git a/src/chrome/src/ui/locales/id.js b/src/chrome/src/ui/locales/id.js index 55cb230ef..fb31409ec 100644 --- a/src/chrome/src/ui/locales/id.js +++ b/src/chrome/src/ui/locales/id.js @@ -782,6 +782,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "Tidak disimpan: teks kosong setelah dibersihkan atau tampaknya berisi rahasia (kata sandi, kunci API, token).", "sp.memory.reason.not_found": "Tidak ada memori tersimpan dengan ID tersebut.", "sp.slash.workflows": "Lihat dan kelola alur kerja tersimpan", + "sp.slash.teach": "Rekam tindakan Anda sebagai alur kerja tersimpan", "sp.slash.run_workflow": "Jalankan alur kerja tersimpan berdasarkan ID", "sp.workflows.run_prompt": "Jalankan alur kerja tersimpan “{name}”", "sp.workflows.parameters_for": "Masukkan parameter untuk “{name}”. Nilai hanya digunakan untuk proses ini.", @@ -797,6 +798,13 @@ export default { "sp.workflows.saved": "Alur kerja “{name}” disimpan.", "sp.workflows.deleted": "Alur kerja “{name}” dihapus.", "sp.workflows.error": "Kesalahan alur kerja: {msg}", + "sp.teach.started": "Mengajarkan “{name}”. Jalankan alur kerja, lalu gunakan /teach --end.", + "sp.teach.status": "Mengajarkan “{name}”: {count} tindakan direkam.", + "sp.teach.inactive": "Mode pengajaran tidak aktif.", + "sp.teach.saved": "Alur kerja “{name}” dipelajari dengan {count} langkah.", + "sp.teach.already_active": "Mode pengajaran sudah aktif di tab ini.", + "sp.teach.agent_running": "Hentikan agen yang sedang berjalan sebelum memulai mode pengajaran.", + "sp.teach.capture_unavailable": "Mode pengajaran tidak dapat merekam halaman ini. Buka atau muat ulang halaman http(s) biasa lalu coba lagi.", "sp.workflows.reason.no_trace": "Tidak ada proses terekam yang berhasil dalam percakapan ini. Aktifkan pelacakan, selesaikan tugas, lalu simpan.", "sp.workflows.reason.no_steps": "Proses tersebut tidak memiliki tindakan yang aman untuk disimpan dan dijalankan ulang.", "sp.workflows.reason.not_found": "Tidak ada alur kerja tersimpan dengan ID tersebut.", diff --git a/src/chrome/src/ui/locales/ja.js b/src/chrome/src/ui/locales/ja.js index 76231ec5d..f4facbe62 100644 --- a/src/chrome/src/ui/locales/ja.js +++ b/src/chrome/src/ui/locales/ja.js @@ -782,6 +782,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "保存されませんでした。整理後のテキストが空か、秘密情報（パスワード、API キー、トークン）が含まれているようです。", "sp.memory.reason.not_found": "その ID の保存済みメモリはありません。", "sp.slash.workflows": "保存済みワークフローを一覧・管理", + "sp.slash.teach": "操作を保存済みワークフローとして記録", "sp.slash.run_workflow": "保存済みワークフローを ID で実行", "sp.workflows.run_prompt": "保存済みワークフロー「{name}」を実行", "sp.workflows.parameters_for": "「{name}」のパラメーターを入力してください。値は今回の実行だけに使用されます。", @@ -797,6 +798,13 @@ export default { "sp.workflows.saved": "ワークフロー「{name}」を保存しました。", "sp.workflows.deleted": "ワークフロー「{name}」を削除しました。", "sp.workflows.error": "ワークフローエラー: {msg}", + "sp.teach.started": "「{name}」を学習中です。操作を完了してから /teach --end を使用してください。", + "sp.teach.status": "「{name}」を学習中：{count} 件の操作を記録しました。", + "sp.teach.inactive": "ティーチャーモードは有効ではありません。", + "sp.teach.saved": "ワークフロー「{name}」を {count} ステップで学習しました。", + "sp.teach.already_active": "このタブではティーチャーモードがすでに有効です。", + "sp.teach.agent_running": "ティーチャーモードを開始する前に実行中のエージェントを停止してください。", + "sp.teach.capture_unavailable": "このページはティーチャーモードで記録できません。通常の http(s) ページを開くか再読み込みして、もう一度お試しください。", "sp.workflows.reason.no_trace": "この会話に成功した記録済み実行がありません。トレースを有効にしてタスクを完了し、保存してください。", "sp.workflows.reason.no_steps": "この実行には安全に保存して再実行できる操作がありません。", "sp.workflows.reason.not_found": "そのIDの保存済みワークフローはありません。", diff --git a/src/chrome/src/ui/locales/ko.js b/src/chrome/src/ui/locales/ko.js index 14faff5be..fde85bf6e 100644 --- a/src/chrome/src/ui/locales/ko.js +++ b/src/chrome/src/ui/locales/ko.js @@ -782,6 +782,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "저장하지 않았습니다. 정리 후 텍스트가 비어 있거나 비밀 정보(비밀번호, API 키, 토큰)가 포함된 것으로 보입니다.", "sp.memory.reason.not_found": "해당 ID의 저장된 메모리가 없습니다.", "sp.slash.workflows": "저장된 워크플로 목록 및 관리", + "sp.slash.teach": "작업을 저장된 워크플로로 기록", "sp.slash.run_workflow": "ID로 저장된 워크플로 실행", "sp.workflows.run_prompt": "저장된 워크플로 “{name}” 실행", "sp.workflows.parameters_for": "“{name}”의 매개변수를 입력하세요. 값은 이번 실행에만 사용됩니다.", @@ -797,6 +798,13 @@ export default { "sp.workflows.saved": "워크플로 “{name}”을 저장했습니다.", "sp.workflows.deleted": "워크플로 “{name}”을 삭제했습니다.", "sp.workflows.error": "워크플로 오류: {msg}", + "sp.teach.started": "“{name}”을 학습 중입니다. 작업을 완료한 다음 /teach --end를 사용하세요.", + "sp.teach.status": "“{name}” 학습 중: {count}개 작업을 캡처했습니다.", + "sp.teach.inactive": "교사 모드가 활성화되어 있지 않습니다.", + "sp.teach.saved": "“{name}” 워크플로를 {count}단계로 학습했습니다.", + "sp.teach.already_active": "이 탭에서 교사 모드가 이미 활성화되어 있습니다.", + "sp.teach.agent_running": "교사 모드를 시작하기 전에 실행 중인 에이전트를 중지하세요.", + "sp.teach.capture_unavailable": "교사 모드에서 이 페이지를 캡처할 수 없습니다. 일반 http(s) 페이지를 열거나 새로고침한 후 다시 시도하세요.", "sp.workflows.reason.no_trace": "이 대화에서 성공한 기록 실행을 찾지 못했습니다. 추적을 켜고 작업을 완료한 다음 저장하세요.", "sp.workflows.reason.no_steps": "이 실행에는 안전하게 저장하여 다시 실행할 수 있는 동작이 없습니다.", "sp.workflows.reason.not_found": "해당 ID의 저장된 워크플로가 없습니다.", diff --git a/src/chrome/src/ui/locales/ms.js b/src/chrome/src/ui/locales/ms.js index 1c57acb8b..5b80ce913 100644 --- a/src/chrome/src/ui/locales/ms.js +++ b/src/chrome/src/ui/locales/ms.js @@ -782,6 +782,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "Tidak disimpan: teks kosong selepas dibersihkan atau kelihatan mengandungi rahsia (kata laluan, kunci API, token).", "sp.memory.reason.not_found": "Tiada memori yang disimpan dengan ID tersebut.", "sp.slash.workflows": "Senarai dan urus aliran kerja tersimpan", + "sp.slash.teach": "Rakam tindakan anda sebagai aliran kerja tersimpan", "sp.slash.run_workflow": "Jalankan aliran kerja tersimpan mengikut ID", "sp.workflows.run_prompt": "Jalankan aliran kerja tersimpan “{name}”", "sp.workflows.parameters_for": "Masukkan parameter untuk “{name}”. Nilai hanya digunakan untuk larian ini.", @@ -797,6 +798,13 @@ export default { "sp.workflows.saved": "Aliran kerja “{name}” disimpan.", "sp.workflows.deleted": "Aliran kerja “{name}” dipadam.", "sp.workflows.error": "Ralat aliran kerja: {msg}", + "sp.teach.started": "Mengajar “{name}”. Lakukan aliran kerja, kemudian gunakan /teach --end.", + "sp.teach.status": "Mengajar “{name}”: {count} tindakan dirakam.", + "sp.teach.inactive": "Mod pengajaran tidak aktif.", + "sp.teach.saved": "Aliran kerja “{name}” dipelajari dengan {count} langkah.", + "sp.teach.already_active": "Mod pengajaran sudah aktif dalam tab ini.", + "sp.teach.agent_running": "Hentikan ejen yang sedang berjalan sebelum memulakan mod pengajaran.", + "sp.teach.capture_unavailable": "Mod pengajaran tidak dapat merakam halaman ini. Buka atau muat semula halaman http(s) biasa dan cuba lagi.", "sp.workflows.reason.no_trace": "Tiada pelaksanaan berjaya yang direkodkan dalam perbualan ini. Aktifkan penjejakan, lengkapkan tugas, kemudian simpannya.", "sp.workflows.reason.no_steps": "Pelaksanaan itu tiada tindakan yang boleh disimpan dengan selamat untuk dimainkan semula.", "sp.workflows.reason.not_found": "Tiada aliran kerja tersimpan dengan ID tersebut.", diff --git a/src/chrome/src/ui/locales/nl.js b/src/chrome/src/ui/locales/nl.js index 368547a43..2b5a09a6f 100644 --- a/src/chrome/src/ui/locales/nl.js +++ b/src/chrome/src/ui/locales/nl.js @@ -124,6 +124,7 @@ export default { 'sp.slash.show_memory': 'Toon opgeslagen gebruikersgeheugen', 'sp.slash.forget_memory': 'Een opgeslagen geheugen vergeten op ID', 'sp.slash.workflows': 'Lijst en beheer opgeslagen workflows', + "sp.slash.teach": "Acties opnemen als opgeslagen workflow", 'sp.slash.run_workflow': 'Voer een opgeslagen workflow uit op ID', 'sp.workflows.run_prompt': 'Opgeslagen workflow "{name}" uitvoeren', 'sp.workflows.parameters_for': 'Voer parameters in voor "{name}". Waarden worden alleen voor deze uitvoering gebruikt.', @@ -361,6 +362,13 @@ export default { 'sp.workflows.saved': 'Workflow "{name}" opgeslagen.', 'sp.workflows.deleted': 'Workflow "{name}" verwijderd.', 'sp.workflows.error': 'Workflowfout: {msg}', + "sp.teach.started": "“{name}” wordt aangeleerd. Voer de workflow uit en gebruik daarna /teach --end.", + "sp.teach.status": "“{name}” wordt aangeleerd: {count} acties vastgelegd.", + "sp.teach.inactive": "De leermodus is niet actief.", + "sp.teach.saved": "Workflow “{name}” met {count} stappen aangeleerd.", + "sp.teach.already_active": "De leermodus is al actief in dit tabblad.", + "sp.teach.agent_running": "Stop de actieve agenttaak voordat je de leermodus start.", + "sp.teach.capture_unavailable": "De leermodus kan deze pagina niet vastleggen. Open of herlaad een gewone http(s)-pagina en probeer opnieuw.", 'sp.workflows.reason.no_trace': 'Er is geen succesvolle opgenomen uitvoering gevonden in dit gesprek...', 'sp.workflows.reason.no_steps': 'Die uitvoering heeft geen acties die veilig kunnen worden opgeslagen voor herhaling.', 'sp.workflows.reason.not_found': 'Geen opgeslagen workflow heeft die ID.', diff --git a/src/chrome/src/ui/locales/pl.js b/src/chrome/src/ui/locales/pl.js index 3c45dd715..759b7439e 100644 --- a/src/chrome/src/ui/locales/pl.js +++ b/src/chrome/src/ui/locales/pl.js @@ -773,6 +773,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "Nie zapisano: po oczyszczeniu tekst jest pusty lub wygląda na zawierający sekret (hasło, klucz API, token).", "sp.memory.reason.not_found": "Nie ma zapisanego wpisu o tym ID.", "sp.slash.workflows": "Wyświetlaj i zarządzaj zapisanymi przepływami pracy", + "sp.slash.teach": "Nagraj działania jako zapisany przepływ pracy", "sp.slash.run_workflow": "Uruchom zapisany przepływ pracy według ID", "sp.workflows.run_prompt": "Uruchom zapisany przepływ pracy „{name}”", "sp.workflows.parameters_for": "Wprowadź parametry dla „{name}”. Wartości zostaną użyte tylko w tym uruchomieniu.", @@ -788,6 +789,13 @@ export default { "sp.workflows.saved": "Zapisano przepływ „{name}”.", "sp.workflows.deleted": "Usunięto przepływ „{name}”.", "sp.workflows.error": "Błąd przepływu pracy: {msg}", + "sp.teach.started": "Uczenie „{name}”. Wykonaj przepływ pracy, a następnie użyj /teach --end.", + "sp.teach.status": "Uczenie „{name}”: zarejestrowano {count} działań.", + "sp.teach.inactive": "Tryb nauczania nie jest aktywny.", + "sp.teach.saved": "Nauczono przepływ „{name}” z {count} krokami.", + "sp.teach.already_active": "Tryb nauczania jest już aktywny w tej karcie.", + "sp.teach.agent_running": "Zatrzymaj aktywne działanie agenta przed uruchomieniem trybu nauczania.", + "sp.teach.capture_unavailable": "Tryb nauczania nie może rejestrować tej strony. Otwórz lub odśwież zwykłą stronę http(s) i spróbuj ponownie.", "sp.workflows.reason.no_trace": "W tej rozmowie nie znaleziono udanego zarejestrowanego wykonania. Włącz śledzenie, ukończ zadanie i zapisz je.", "sp.workflows.reason.no_steps": "To wykonanie nie zawiera działań, które można bezpiecznie zapisać do ponownego użycia.", "sp.workflows.reason.not_found": "Nie ma zapisanego przepływu pracy o tym ID.", diff --git a/src/chrome/src/ui/locales/pt.js b/src/chrome/src/ui/locales/pt.js index e858d1926..1757bf718 100644 --- a/src/chrome/src/ui/locales/pt.js +++ b/src/chrome/src/ui/locales/pt.js @@ -128,6 +128,7 @@ export default { 'sp.slash.show_memory': "Mostrar memória salva do usuário", 'sp.slash.forget_memory': "Esqueça uma memória salva por ID", 'sp.slash.workflows': "Liste e gerencie fluxos de trabalho salvos", + "sp.slash.teach": "Grave suas ações como um fluxo de trabalho salvo", 'sp.slash.run_workflow': "Execute um fluxo de trabalho salvo por ID", 'sp.workflows.run_prompt': "Execute o fluxo de trabalho salvo “{name}”", 'sp.workflows.parameters_for': "Insira os parâmetros para “{name}”. Os valores são usados ​​apenas para esta execução.", @@ -375,6 +376,13 @@ export default { 'sp.workflows.saved': "Fluxo de trabalho salvo “{name}”.", 'sp.workflows.deleted': "Fluxo de trabalho “{name}” excluído.", 'sp.workflows.error': "Erro de fluxo de trabalho: {msg}", + "sp.teach.started": "Ensinando “{name}”. Execute o fluxo e use /teach --end.", + "sp.teach.status": "Ensinando “{name}”: {count} ações capturadas.", + "sp.teach.inactive": "O modo de ensino não está ativo.", + "sp.teach.saved": "Fluxo “{name}” aprendido com {count} etapas.", + "sp.teach.already_active": "O modo de ensino já está ativo nesta aba.", + "sp.teach.agent_running": "Interrompa a execução ativa do agente antes de iniciar o modo de ensino.", + "sp.teach.capture_unavailable": "O modo de ensino não pode capturar esta página. Abra ou recarregue uma página http(s) comum e tente novamente.", 'sp.workflows.reason.no_trace': "Nenhuma execução gravada com sucesso foi encontrada nesta conversa. Ative o rastreamento, conclua a tarefa e salve-a.", 'sp.workflows.reason.no_steps': "Essa execução não possui ações que possam ser salvas com segurança para repetição.", 'sp.workflows.reason.not_found': "Nenhum fluxo de trabalho salvo possui esse ID.", diff --git a/src/chrome/src/ui/locales/ru.js b/src/chrome/src/ui/locales/ru.js index a33d2f36a..6ef05033e 100644 --- a/src/chrome/src/ui/locales/ru.js +++ b/src/chrome/src/ui/locales/ru.js @@ -782,6 +782,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "Не сохранено: после очистки текст пуст или похож на секрет (пароль, API-ключ, токен).", "sp.memory.reason.not_found": "Сохранённой записи с таким ID нет.", "sp.slash.workflows": "Просмотр и управление сохранёнными сценариями", + "sp.slash.teach": "Записать ваши действия как сохранённый рабочий процесс", "sp.slash.run_workflow": "Запустить сохранённый сценарий по ID", "sp.workflows.run_prompt": "Запустить сохранённый сценарий «{name}»", "sp.workflows.parameters_for": "Введите параметры для «{name}». Значения используются только в этом запуске.", @@ -797,6 +798,13 @@ export default { "sp.workflows.saved": "Сценарий «{name}» сохранён.", "sp.workflows.deleted": "Сценарий «{name}» удалён.", "sp.workflows.error": "Ошибка сценария: {msg}", + "sp.teach.started": "Обучение «{name}». Выполните процесс, затем используйте /teach --end.", + "sp.teach.status": "Обучение «{name}»: записано действий — {count}.", + "sp.teach.inactive": "Режим обучения не активен.", + "sp.teach.saved": "Рабочий процесс «{name}» изучен: {count} шагов.", + "sp.teach.already_active": "Режим обучения уже активен на этой вкладке.", + "sp.teach.agent_running": "Остановите активный запуск агента перед включением режима обучения.", + "sp.teach.capture_unavailable": "Режим обучения не может записывать эту страницу. Откройте или перезагрузите обычную страницу http(s) и повторите попытку.", "sp.workflows.reason.no_trace": "В этом диалоге нет успешного записанного запуска. Включите трассировку, завершите задачу и сохраните её.", "sp.workflows.reason.no_steps": "В этом запуске нет действий, которые можно безопасно сохранить для повтора.", "sp.workflows.reason.not_found": "Сохранённого сценария с таким ID нет.", diff --git a/src/chrome/src/ui/locales/th.js b/src/chrome/src/ui/locales/th.js index 2ff2e15e1..c5d588ae9 100644 --- a/src/chrome/src/ui/locales/th.js +++ b/src/chrome/src/ui/locales/th.js @@ -782,6 +782,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "ไม่ได้บันทึก: ข้อความว่างหลังการล้างหรือดูเหมือนมีข้อมูลลับ (รหัสผ่าน คีย์ API โทเค็น)", "sp.memory.reason.not_found": "ไม่มีหน่วยความจำที่บันทึกไว้ด้วย ID นี้", "sp.slash.workflows": "แสดงและจัดการเวิร์กโฟลว์ที่บันทึกไว้", + "sp.slash.teach": "บันทึกการทำงานของคุณเป็นเวิร์กโฟลว์ที่บันทึกไว้", "sp.slash.run_workflow": "เรียกใช้เวิร์กโฟลว์ที่บันทึกไว้ด้วย ID", "sp.workflows.run_prompt": "เรียกใช้เวิร์กโฟลว์ที่บันทึกไว้ “{name}”", "sp.workflows.parameters_for": "ป้อนพารามิเตอร์สำหรับ “{name}” ค่าจะใช้สำหรับการเรียกใช้ครั้งนี้เท่านั้น", @@ -797,6 +798,13 @@ export default { "sp.workflows.saved": "บันทึกเวิร์กโฟลว์ “{name}” แล้ว", "sp.workflows.deleted": "ลบเวิร์กโฟลว์ “{name}” แล้ว", "sp.workflows.error": "ข้อผิดพลาดของเวิร์กโฟลว์: {msg}", + "sp.teach.started": "กำลังสอน “{name}” ทำเวิร์กโฟลว์ให้เสร็จ แล้วใช้ /teach --end", + "sp.teach.status": "กำลังสอน “{name}”: บันทึกแล้ว {count} การทำงาน", + "sp.teach.inactive": "โหมดสอนไม่ได้เปิดอยู่", + "sp.teach.saved": "เรียนรู้เวิร์กโฟลว์ “{name}” แล้ว {count} ขั้นตอน", + "sp.teach.already_active": "โหมดสอนเปิดอยู่แล้วในแท็บนี้", + "sp.teach.agent_running": "หยุดการทำงานของเอเจนต์ก่อนเริ่มโหมดสอน", + "sp.teach.capture_unavailable": "โหมดสอนไม่สามารถบันทึกหน้านี้ได้ เปิดหรือโหลดหน้า http(s) ปกติใหม่แล้วลองอีกครั้ง", "sp.workflows.reason.no_trace": "ไม่พบการทำงานที่บันทึกและสำเร็จในการสนทนานี้ เปิดการติดตาม ทำงานให้เสร็จ แล้วบันทึก", "sp.workflows.reason.no_steps": "การทำงานนี้ไม่มีการกระทำที่สามารถบันทึกเพื่อเล่นซ้ำได้อย่างปลอดภัย", "sp.workflows.reason.not_found": "ไม่มีเวิร์กโฟลว์ที่บันทึกไว้ด้วย ID นี้", diff --git a/src/chrome/src/ui/locales/tl.js b/src/chrome/src/ui/locales/tl.js index 94cb00c13..13dc716c4 100644 --- a/src/chrome/src/ui/locales/tl.js +++ b/src/chrome/src/ui/locales/tl.js @@ -782,6 +782,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "Hindi na-save: walang laman ang text pagkatapos linisin o mukhang may lihim (password, API key, token).", "sp.memory.reason.not_found": "Walang naka-save na memory na may ganoong ID.", "sp.slash.workflows": "Ilista at pamahalaan ang mga naka-save na workflow", + "sp.slash.teach": "I-record ang iyong mga kilos bilang naka-save na workflow", "sp.slash.run_workflow": "Patakbuhin ang naka-save na workflow ayon sa ID", "sp.workflows.run_prompt": "Patakbuhin ang naka-save na workflow na “{name}”", "sp.workflows.parameters_for": "Ilagay ang mga parameter para sa “{name}”. Para sa run na ito lang gagamitin ang mga value.", @@ -797,6 +798,13 @@ export default { "sp.workflows.saved": "Na-save ang workflow na “{name}”.", "sp.workflows.deleted": "Tinanggal ang workflow na “{name}”.", "sp.workflows.error": "Error sa workflow: {msg}", + "sp.teach.started": "Itinuturo ang “{name}”. Gawin ang workflow, pagkatapos ay gamitin ang /teach --end.", + "sp.teach.status": "Itinuturo ang “{name}”: {count} kilos ang nakuha.", + "sp.teach.inactive": "Hindi aktibo ang teacher mode.", + "sp.teach.saved": "Natutunan ang workflow na “{name}” na may {count} hakbang.", + "sp.teach.already_active": "Aktibo na ang teacher mode sa tab na ito.", + "sp.teach.agent_running": "Itigil ang aktibong agent run bago simulan ang teacher mode.", + "sp.teach.capture_unavailable": "Hindi makuha ng teacher mode ang pahinang ito. Magbukas o mag-reload ng karaniwang http(s) page at subukan muli.", "sp.workflows.reason.no_trace": "Walang matagumpay na naitalang run sa usapang ito. I-on ang tracing, tapusin ang gawain, at i-save ito.", "sp.workflows.reason.no_steps": "Walang aksyon sa run na ito na ligtas i-save para sa muling pagpapatakbo.", "sp.workflows.reason.not_found": "Walang naka-save na workflow na may ganitong ID.", diff --git a/src/chrome/src/ui/locales/tr.js b/src/chrome/src/ui/locales/tr.js index dde0d3684..4508ac521 100644 --- a/src/chrome/src/ui/locales/tr.js +++ b/src/chrome/src/ui/locales/tr.js @@ -781,6 +781,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "Kaydedilmedi: temizleme sonrasında metin boş kalıyor veya bir gizli bilgi (parola, API anahtarı, token) içeriyor gibi görünüyor.", "sp.memory.reason.not_found": "Bu kimliğe sahip kaydedilmiş bellek yok.", "sp.slash.workflows": "Kayıtlı iş akışlarını listele ve yönet", + "sp.slash.teach": "Eylemlerinizi kayıtlı bir iş akışı olarak kaydedin", "sp.slash.run_workflow": "Kaydedilmiş iş akışını kimliğe göre çalıştır", "sp.workflows.run_prompt": "Kaydedilmiş “{name}” iş akışını çalıştır", "sp.workflows.parameters_for": "“{name}” için parametreleri girin. Değerler yalnızca bu çalıştırmada kullanılır.", @@ -796,6 +797,13 @@ export default { "sp.workflows.saved": "“{name}” iş akışı kaydedildi.", "sp.workflows.deleted": "“{name}” iş akışı silindi.", "sp.workflows.error": "İş akışı hatası: {msg}", + "sp.teach.started": "“{name}” öğretiliyor. İş akışını tamamlayın, ardından /teach --end kullanın.", + "sp.teach.status": "“{name}” öğretiliyor: {count} eylem yakalandı.", + "sp.teach.inactive": "Öğretme modu etkin değil.", + "sp.teach.saved": "“{name}” iş akışı {count} adımla öğrenildi.", + "sp.teach.already_active": "Öğretme modu bu sekmede zaten etkin.", + "sp.teach.agent_running": "Öğretme modunu başlatmadan önce etkin ajan çalışmasını durdurun.", + "sp.teach.capture_unavailable": "Öğretme modu bu sayfayı yakalayamıyor. Normal bir http(s) sayfası açın veya yenileyip tekrar deneyin.", "sp.workflows.reason.no_trace": "Bu konuşmada başarılı ve kaydedilmiş bir çalışma bulunamadı. İzlemeyi açın, görevi tamamlayın ve kaydedin.", "sp.workflows.reason.no_steps": "Bu çalışmada güvenle kaydedilip yeniden oynatılabilecek bir eylem yok.", "sp.workflows.reason.not_found": "Bu kimliğe sahip kayıtlı iş akışı yok.", diff --git a/src/chrome/src/ui/locales/uk.js b/src/chrome/src/ui/locales/uk.js index 68d5880f8..b05278008 100644 --- a/src/chrome/src/ui/locales/uk.js +++ b/src/chrome/src/ui/locales/uk.js @@ -782,6 +782,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "Не збережено: після очищення текст порожній або схожий на секрет (пароль, ключ API, токен).", "sp.memory.reason.not_found": "Збереженого запису з таким ID немає.", "sp.slash.workflows": "Перегляд і керування збереженими сценаріями", + "sp.slash.teach": "Записати ваші дії як збережений робочий процес", "sp.slash.run_workflow": "Запустити збережений сценарій за ID", "sp.workflows.run_prompt": "Запустити збережений сценарій «{name}»", "sp.workflows.parameters_for": "Введіть параметри для «{name}». Значення використовуються лише для цього запуску.", @@ -797,6 +798,13 @@ export default { "sp.workflows.saved": "Сценарій «{name}» збережено.", "sp.workflows.deleted": "Сценарій «{name}» видалено.", "sp.workflows.error": "Помилка сценарію: {msg}", + "sp.teach.started": "Навчання «{name}». Виконайте процес, потім скористайтеся /teach --end.", + "sp.teach.status": "Навчання «{name}»: записано дій — {count}.", + "sp.teach.inactive": "Режим навчання не активний.", + "sp.teach.saved": "Робочий процес «{name}» вивчено: {count} кроків.", + "sp.teach.already_active": "Режим навчання вже активний у цій вкладці.", + "sp.teach.agent_running": "Зупиніть активний запуск агента перед увімкненням режиму навчання.", + "sp.teach.capture_unavailable": "Режим навчання не може записувати цю сторінку. Відкрийте або перезавантажте звичайну сторінку http(s) і спробуйте ще раз.", "sp.workflows.reason.no_trace": "У цій розмові не знайдено успішного записаного запуску. Увімкніть трасування, завершіть завдання та збережіть його.", "sp.workflows.reason.no_steps": "Цей запуск не містить дій, які можна безпечно зберегти для повтору.", "sp.workflows.reason.not_found": "Немає збереженого сценарію з таким ID.", diff --git a/src/chrome/src/ui/locales/vi.js b/src/chrome/src/ui/locales/vi.js index 2ed022332..e9e2fca44 100644 --- a/src/chrome/src/ui/locales/vi.js +++ b/src/chrome/src/ui/locales/vi.js @@ -128,6 +128,7 @@ export default { 'sp.slash.show_memory': "Hiển thị bộ nhớ người dùng đã lưu", 'sp.slash.forget_memory': "Quên bộ nhớ đã lưu bằng ID", 'sp.slash.workflows': "Liệt kê và quản lý quy trình công việc đã lưu", + "sp.slash.teach": "Ghi lại thao tác của bạn thành quy trình đã lưu", 'sp.slash.run_workflow': "Chạy quy trình công việc đã lưu theo ID", 'sp.workflows.run_prompt': "Chạy quy trình làm việc đã lưu “{name}”", 'sp.workflows.parameters_for': "Nhập thông số cho “{name}”. Các giá trị chỉ được sử dụng cho lần chạy này.", @@ -375,6 +376,13 @@ export default { 'sp.workflows.saved': "Quy trình công việc đã lưu “{name}”.", 'sp.workflows.deleted': "Đã xóa quy trình công việc “{name}”.", 'sp.workflows.error': "Lỗi quy trình làm việc: {msg}", + "sp.teach.started": "Đang dạy “{name}”. Hãy thực hiện quy trình rồi dùng /teach --end.", + "sp.teach.status": "Đang dạy “{name}”: đã ghi {count} thao tác.", + "sp.teach.inactive": "Chế độ dạy chưa hoạt động.", + "sp.teach.saved": "Đã học quy trình “{name}” gồm {count} bước.", + "sp.teach.already_active": "Chế độ dạy đã hoạt động trong thẻ này.", + "sp.teach.agent_running": "Hãy dừng lượt chạy tác nhân trước khi bắt đầu chế độ dạy.", + "sp.teach.capture_unavailable": "Chế độ dạy không thể ghi lại trang này. Hãy mở hoặc tải lại một trang http(s) thông thường rồi thử lại.", 'sp.workflows.reason.no_trace': "Không tìm thấy lần chạy nào được ghi thành công trong cuộc trò chuyện này. Kích hoạt tính năng theo dõi, hoàn thành nhiệm vụ rồi lưu lại.", 'sp.workflows.reason.no_steps': "Lần chạy đó không có hành động nào có thể được lưu lại một cách an toàn để phát lại.", 'sp.workflows.reason.not_found': "Không có quy trình làm việc đã lưu nào có ID đó.", diff --git a/src/chrome/src/ui/locales/zh.js b/src/chrome/src/ui/locales/zh.js index 4fb9a7dd5..734ca24b4 100644 --- a/src/chrome/src/ui/locales/zh.js +++ b/src/chrome/src/ui/locales/zh.js @@ -782,6 +782,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "未保存：清理后的文本为空，或看起来包含机密信息（密码、API 密钥、令牌）。", "sp.memory.reason.not_found": "没有使用该 ID 的已保存记忆。", "sp.slash.workflows": "列出和管理已保存的工作流", + "sp.slash.teach": "将您的操作记录为已保存的工作流", "sp.slash.run_workflow": "按 ID 运行已保存的工作流", "sp.workflows.run_prompt": "运行已保存的工作流“{name}”", "sp.workflows.parameters_for": "请输入“{name}”的参数。参数值仅用于本次运行。", @@ -797,6 +798,13 @@ export default { "sp.workflows.saved": "已保存工作流“{name}”。", "sp.workflows.deleted": "已删除工作流“{name}”。", "sp.workflows.error": "工作流错误：{msg}", + "sp.teach.started": "正在示教“{name}”。请完成工作流，然后使用 /teach --end。", + "sp.teach.status": "正在示教“{name}”：已捕获 {count} 个操作。", + "sp.teach.inactive": "示教模式未启用。", + "sp.teach.saved": "已学习工作流“{name}”，共 {count} 个步骤。", + "sp.teach.already_active": "此标签页已启用示教模式。", + "sp.teach.agent_running": "请先停止当前 Agent 运行，再启动示教模式。", + "sp.teach.capture_unavailable": "示教模式无法捕获此页面。请打开或重新加载普通 http(s) 页面后重试。", "sp.workflows.reason.no_trace": "当前对话中没有成功且已记录的运行。请启用跟踪、完成任务，然后保存。", "sp.workflows.reason.no_steps": "该运行没有可以安全保存并重放的操作。", "sp.workflows.reason.not_found": "没有使用该 ID 的已保存工作流。", diff --git a/src/chrome/src/ui/sidepanel.js b/src/chrome/src/ui/sidepanel.js index b5644c7b4..469d2da2f 100644 --- a/src/chrome/src/ui/sidepanel.js +++ b/src/chrome/src/ui/sidepanel.js @@ -633,6 +633,17 @@ const SLASH_COMMANDS = [ { value: '--file', descriptionKey: 'sp.slash.import_config_file', disallowPayload: true, requires: '--import' }, ], }, + { + value: '/teach', + usage: '/teach [--start | --end]', + descriptionKey: 'sp.slash.teach', + action: 'status', + outOfBand: true, + options: [ + { value: '--start', valueLabel: '', descriptionKey: 'sp.slash.teach', action: 'start', takesRemainder: true, outOfBand: true, exclusiveGroup: 'teach-action' }, + { value: '--end', descriptionKey: 'sp.slash.teach', action: 'end', disallowPayload: true, outOfBand: true, exclusiveGroup: 'teach-action' }, + ], + }, { value: '/allow-api', usage: '/allow-api [prompt]', descriptionKey: 'sp.slash.allow_api', action: 'enable', acceptsPayload: true }, { value: '/foreground', usage: '/foreground [prompt]', descriptionKey: 'sp.slash.foreground', action: 'enable', acceptsPayload: true }, { value: '/dangerously-skip-permissions', usage: '/dangerously-skip-permissions [prompt]', descriptionKey: 'sp.slash.dangerously_skip_permissions', action: 'disable', acceptsPayload: true, outOfBand: true }, @@ -3530,6 +3541,60 @@ function bindSavedWorkflowManager(manager) { }); } +function teacherModeFailureMessage(res) { + if (res?.reason === 'already_active') return t('sp.teach.already_active'); + if (res?.reason === 'no_active_session') return t('sp.teach.inactive'); + if (res?.reason === 'agent_running') return t('sp.teach.agent_running'); + if (res?.reason === 'capture_unavailable') return t('sp.teach.capture_unavailable'); + return savedWorkflowFailureMessage(res); +} + +async function showTeacherModeStatus(tabId = currentTabId) { + try { + const res = await sendToBackground('get_teacher_mode', { tabId }); + if (currentTabId !== tabId) return; + if (!res?.ok) throw new Error(res?.error || res?.reason || 'unknown error'); + addPersistentSlashMessage(res.session?.active + ? t('sp.teach.status', { name: res.session.name, count: res.session.actionCount || 0 }) + : t('sp.teach.inactive')); + } catch (error) { + if (currentTabId === tabId) showComposerToast(t('sp.workflows.error', { msg: error.message }), { duration: 7000 }); + } +} + +async function startTeacherMode(name, tabId = currentTabId) { + try { + const res = await sendToBackground('start_teacher_mode', { tabId, name }); + if (currentTabId !== tabId) return; + if (!res?.ok) { + showComposerToast(teacherModeFailureMessage(res), { duration: 7000 }); + return; + } + addPersistentSlashMessage(t('sp.teach.started', { name: res.session?.name || name })); + } catch (error) { + if (currentTabId === tabId) showComposerToast(t('sp.workflows.error', { msg: error.message }), { duration: 7000 }); + } +} + +async function endTeacherMode(tabId = currentTabId) { + try { + const res = await sendToBackground('end_teacher_mode', { tabId }); + if (currentTabId !== tabId) return; + if (!res?.ok) { + showComposerToast(teacherModeFailureMessage(res), { duration: 7000 }); + return; + } + const message = t('sp.teach.saved', { + name: res.workflow?.name || '', + count: res.workflow?.steps?.length || 0, + }); + const warnings = Array.isArray(res.warnings) ? res.warnings.filter(Boolean) : []; + addPersistentSlashMessage(warnings.length ? `${message} ${warnings.join(' ')}` : message); + } catch (error) { + if (currentTabId === tabId) showComposerToast(t('sp.workflows.error', { msg: error.message }), { duration: 7000 }); + } +} + async function showSavedWorkflows(tabId = currentTabId) { try { const res = await sendToBackground('list_saved_workflows'); @@ -6985,6 +7050,21 @@ async function parseSlashCommands(text, tabId = currentTabId, options = {}) { return ''; } + if (command.value === '/teach' && action === 'status') { + await showTeacherModeStatus(tabId); + return ''; + } + + if (command.value === '/teach' && action === 'start') { + await startTeacherMode(payload, tabId); + return ''; + } + + if (command.value === '/teach' && action === 'end') { + await endTeacherMode(tabId); + return ''; + } + if (command.value === '/workflow' && action === 'list') { await showSavedWorkflows(tabId); return ''; diff --git a/src/firefox/manifest.json b/src/firefox/manifest.json index b9086a8e1..7daca097b 100644 --- a/src/firefox/manifest.json +++ b/src/firefox/manifest.json @@ -42,6 +42,7 @@ "js": [ "src/content/rich-text-toolbar-heuristic.js", "src/content/accessibility-tree.js", + "src/content/teacher-capture.js", "src/content/content.js", "src/content/agent-visual-indicator.js", "src/content/ollama-launch-handoff.js" diff --git a/src/firefox/src/agent/agent.js b/src/firefox/src/agent/agent.js index 8e99dc325..7fec775e1 100644 --- a/src/firefox/src/agent/agent.js +++ b/src/firefox/src/agent/agent.js @@ -434,6 +434,7 @@ export class Agent extends LoopDetector { this._recentSubmitClicks = new Map(); this._formValidationBlocks = new Map(); // tabId -> validation state that must change before another submit this._runningTabs = new Set(); // tabIds with an active processMessage/Stream in flight + this._runStartGuard = null; this.completionInvariants = new Map(); // tabId -> run-scoped post-action verification state this._completionRunCounter = 0; this.scheduler = null; @@ -820,6 +821,38 @@ export class Agent extends LoopDetector { this._conversationScopeChangeListener = typeof listener === 'function' ? listener : null; } + setRunStartGuard(guard) { + this._runStartGuard = typeof guard === 'function' ? guard : null; + } + + _runEntryKind(defaultKind, runOptions = {}) { + if (runOptions?.cloudRun === true) return 'cloud'; + if (runOptions?.scheduledRun === true) return 'scheduled'; + return defaultKind; + } + + async assertRunStartAllowed(tabId, defaultKind = 'interactive', runOptions = {}) { + await this._runStartGuard?.(tabId, { + kind: this._runEntryKind(defaultKind, runOptions), + runOptions, + }); + } + + async _claimRunEntry(tabId, defaultKind, runOptions = {}) { + if (this._runningTabs.has(tabId)) { + throw new Error('An agent run is already in progress for this tab.'); + } + // Claim synchronously before awaiting the external guard so teacher-mode + // startup cannot race a run whose persisted teacher-state check is pending. + this._runningTabs.add(tabId); + try { + await this.assertRunStartAllowed(tabId, defaultKind, runOptions); + } catch (error) { + this._runningTabs.delete(tabId); + throw error; + } + } + isRunning(tabId) { return this._runningTabs.has(tabId); } @@ -13815,44 +13848,59 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d return originalResponse; } + async _endSavedWorkflowTraceRun(runId, status, finalContent) { + await trace.endRun(runId, { status, finalContent }); + } + async replaySavedWorkflow(tabId, workflow, parameters = {}, onUpdate = () => {}, runOptions = {}) { - if (this._runningTabs.has(tabId)) throw new Error('An agent run is already in progress for this tab.'); if (!workflow?.id || !Array.isArray(workflow.steps) || !workflow.steps.length) { throw new Error('Saved workflow is missing or invalid.'); } - await this._hydrate(tabId); - // Deterministic workflow replay is an independent task even when it never - // falls back to processMessage. Clear stale selected-text grounding before - // any replay action so a successful replay cannot poison the next turn. - this._clearSelectionGroundingForIndependentRun(tabId, { ...runOptions, independentRun: true }); - // Align pre-run cleanup with processMessage so a prior Act turn cannot - // leak plan guards or active-skill state into deterministic replay (or - // the reverse on the next turn). Chrome-only CDP fallback state is - // optional here. - this._resetActiveSkillsForRun(tabId, { refreshPrompt: false }); - this._clearRunLoopState(tabId); - this._resetRichTextToolbarAudit(tabId); - this._clickAxCdpFallbacks?.delete(tabId); - this.abortFlags.delete(tabId); - this._prepareClarificationAuthorizationForRun(tabId); - this.permissions.beginTurn(tabId); - this.conversationModes.set(tabId, 'act'); - const completionRunToken = this._beginCompletionInvariant(tabId); - this._runningTabs.add(tabId); - const startUrl = await this._currentUrl(tabId); - const conversationId = await this.ensureConversationId(tabId, 'act'); - const traceRunId = await trace.startRun({ - conversationId, - userMessage: `Run saved workflow: ${workflow.name}`, - tabUrl: startUrl, - mode: 'act', - model: this.providerManager?.getActive?.()?.model || '', - providerId: this.providerManager?.activeProviderId || '', - runtimeConfig: this._runtimeTraceConfig(this.providerManager?.getActive?.(), { - tabId, + await this._claimRunEntry(tabId, 'workflow', runOptions); + let completionRunToken = ''; + let startUrl = ''; + let traceRunId = null; + try { + await this._hydrate(tabId); + // Deterministic workflow replay is an independent task even when it never + // falls back to processMessage. Clear stale selected-text grounding before + // any replay action so a successful replay cannot poison the next turn. + this._clearSelectionGroundingForIndependentRun(tabId, { ...runOptions, independentRun: true }); + // Align pre-run cleanup with processMessage so a prior Act turn cannot + // leak plan guards or active-skill state into deterministic replay (or + // the reverse on the next turn). Chrome-only CDP fallback state is + // optional here. + this._resetActiveSkillsForRun(tabId, { refreshPrompt: false }); + this._clearRunLoopState(tabId); + this._resetRichTextToolbarAudit(tabId); + this._clickAxCdpFallbacks?.delete(tabId); + this.abortFlags.delete(tabId); + this._prepareClarificationAuthorizationForRun(tabId); + this.permissions.beginTurn(tabId); + this.conversationModes.set(tabId, 'act'); + completionRunToken = this._beginCompletionInvariant(tabId); + startUrl = await this._currentUrl(tabId); + const conversationId = await this.ensureConversationId(tabId, 'act'); + traceRunId = await trace.startRun({ + conversationId, + userMessage: `Run saved workflow: ${workflow.name}`, + tabUrl: startUrl, mode: 'act', - }), - }); + model: this.providerManager?.getActive?.()?.model || '', + providerId: this.providerManager?.activeProviderId || '', + runtimeConfig: this._runtimeTraceConfig(this.providerManager?.getActive?.(), { + tabId, + mode: 'act', + }), + }); + } catch (error) { + try { + if (completionRunToken) this._clearCompletionInvariant(tabId, completionRunToken); + } finally { + this._runningTabs.delete(tabId); + } + throw error; + } let traceStatus = 'workflow_stopped'; let finalContent = ''; let matchedSteps = 0; @@ -14098,15 +14146,24 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d healings: verifiedHealings, }; } finally { - await trace.endRun(traceRunId, { status: traceStatus, finalContent }); - this.currentCostState.delete(tabId); - this._planExecutionGuards.delete(tabId); - this._resetActiveSkillsForRun(tabId); - this._runningTabs.delete(tabId); - this._clearRunLoopState(tabId); - if (traceStatus !== 'workflow_fallback') this._resetRichTextToolbarAudit(tabId); - this._clickAxCdpFallbacks?.delete(tabId); - this._clearCompletionInvariant(tabId, completionRunToken); + try { + try { + await this._endSavedWorkflowTraceRun(traceRunId, traceStatus, finalContent); + } finally { + this.currentCostState.delete(tabId); + this._planExecutionGuards.delete(tabId); + this._resetActiveSkillsForRun(tabId); + this._clearRunLoopState(tabId); + if (traceStatus !== 'workflow_fallback') this._resetRichTextToolbarAudit(tabId); + this._clickAxCdpFallbacks?.delete(tabId); + } + } finally { + try { + this._clearCompletionInvariant(tabId, completionRunToken); + } finally { + this._runningTabs.delete(tabId); + } + } } } @@ -16901,10 +16958,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d * @returns {Promise} final text response */ async processMessage(tabId, userMessage, onUpdate = () => {}, mode = 'ask', attachments = [], runOptions = {}) { - if (this._runningTabs.has(tabId)) { - throw new Error('An agent run is already in progress for this tab.'); - } - this._runningTabs.add(tabId); + await this._claimRunEntry(tabId, 'interactive', runOptions); try { // Hydration has to run before the toolbar-ledger reset below, so a // persisted obligation cannot outlive the run that cleared it. It is @@ -16924,7 +16978,6 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d } if (runOptions?.trustedContinuation !== true) this._continuationExecutionEvidence.delete(tabId); const completionRunToken = this._beginCompletionInvariant(tabId); - this._runningTabs.add(tabId); if (runOptions?.trustedContinuation !== true) this.persistenceDegradedTabs.delete(tabId); this._runUpdateCallbacks.set(tabId, onUpdate); this._runModeOverrides.set(tabId, mode); @@ -17802,10 +17855,7 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d * Process a message with streaming output. */ async processMessageStream(tabId, userMessage, onUpdate = () => {}, mode = 'ask', runOptions = {}) { - if (this._runningTabs.has(tabId)) { - throw new Error('An agent run is already in progress for this tab.'); - } - this._runningTabs.add(tabId); + await this._claimRunEntry(tabId, 'interactive', runOptions); try { // Hydration has to run before the toolbar-ledger reset below, so a // persisted obligation cannot outlive the run that cleared it. It is @@ -17825,7 +17875,6 @@ Rules: no prose intro, no conclusion, no "this screenshot shows...", no layout d } if (runOptions?.trustedContinuation !== true) this._continuationExecutionEvidence.delete(tabId); const completionRunToken = this._beginCompletionInvariant(tabId); - this._runningTabs.add(tabId); if (runOptions?.trustedContinuation !== true) this.persistenceDegradedTabs.delete(tabId); this._runUpdateCallbacks.set(tabId, onUpdate); this._runModeOverrides.set(tabId, mode); diff --git a/src/firefox/src/agent/scheduler.js b/src/firefox/src/agent/scheduler.js index 12a6591c9..ee7f4840b 100644 --- a/src/firefox/src/agent/scheduler.js +++ b/src/firefox/src/agent/scheduler.js @@ -566,6 +566,10 @@ export class ScheduledJobManager { this.restoreAlarms().catch((e) => console.warn('[WebBrain] restore scheduled alarms failed:', e)); } + isRunning(tabId) { + return this._runningTabs.has(tabId); + } + async _getJobs() { const stored = await this.api.storage.local.get(SCHEDULED_JOBS_KEY); const jobs = stored?.[SCHEDULED_JOBS_KEY]; @@ -1499,20 +1503,49 @@ export class ScheduledJobManager { } let tabId; + let reservedTabId = null; + const releaseReservation = () => { + if (reservedTabId == null) return; + this._runningTabs.delete(reservedTabId); + reservedTabId = null; + }; + const reserveTab = async (candidateTabId) => { + if (candidateTabId == null) return; + const numericTabId = Number(candidateTabId); + const resolvedTabId = Number.isFinite(numericTabId) ? numericTabId : candidateTabId; + if (this._runningTabs.has(resolvedTabId) || this.agent.isRunning(resolvedTabId)) { + throw new Error('An agent run is already in progress for this tab.'); + } + this._runningTabs.add(resolvedTabId); + reservedTabId = resolvedTabId; + try { + await this.agent.assertRunStartAllowed?.(resolvedTabId, 'scheduled', { scheduledRun: true }); + } catch (error) { + releaseReservation(); + throw error; + } + }; try { + const candidateTabId = job.kind === 'resume' || job.target?.type === 'current_tab' + ? (job.tabId || job.target?.tabId) + : job.target?.tabId; + await reserveTab(candidateTabId); tabId = await this._resolveTab(job); + if (reservedTabId !== tabId) { + releaseReservation(); + await reserveTab(tabId); + } await this._validateConversation(job, tabId); await this._validateTaskTarget(job, tabId); } catch (e) { - await this._markFailed(job, e.message); - return; - } - - if (this._runningTabs.has(tabId) || this.agent.isRunning(tabId)) { - await this._requeue(job, 'The target tab already has an active WebBrain run.'); + releaseReservation(); + if (isActiveRunError(e) || e?.code === 'teacher_mode_active') { + await this._requeue(job, 'The target tab already has an active WebBrain run.'); + } else { + await this._markFailed(job, e.message); + } return; } - this._runningTabs.add(tabId); const running = await this._updateJobIf(job.id, (prev) => ( ['pending', 'queued'].includes(prev.status) diff --git a/src/firefox/src/agent/teacher-mode.js b/src/firefox/src/agent/teacher-mode.js new file mode 100644 index 000000000..ea725ea14 --- /dev/null +++ b/src/firefox/src/agent/teacher-mode.js @@ -0,0 +1,226 @@ +import { + normalizeSavedWorkflowName, + normalizeTeacherCaptureAction, + workflowUrlScope, +} from './workflows.js'; + +export const TEACHER_SESSION_PREFIX = 'teacherSession:'; +export const TEACHER_ACTION_LIMIT = 100; +export const TEACHER_RUN_CONFLICT_CODE = 'teacher_mode_active'; + +function teacherRunConflictError() { + const error = new Error('Teacher mode is active for this tab. End teacher mode before starting an automated run.'); + error.code = TEACHER_RUN_CONFLICT_CODE; + return error; +} + +export function createTeacherRunInterlock(store, options = {}) { + const automationOwnsTab = typeof options.automationOwnsTab === 'function' + ? options.automationOwnsTab + : () => false; + let lock = Promise.resolve(); + + const withLock = async (task) => { + const run = lock.then(task, task); + lock = run.catch(() => {}); + return run; + }; + + const blockedResult = async (tabId) => ({ + changed: false, + reason: 'agent_running', + session: await store.get(tabId), + }); + + return { + withLock, + async guardRunStart(tabId) { + return withLock(async () => { + if (await store.get(tabId)) throw teacherRunConflictError(); + }); + }, + async start(tabId, input = {}) { + return withLock(async () => { + if (automationOwnsTab(tabId)) return blockedResult(tabId); + return store.start(tabId, input); + }); + }, + async record(tabId, action) { + return withLock(async () => { + if (automationOwnsTab(tabId)) return blockedResult(tabId); + return store.record(tabId, action); + }); + }, + async navigation(tabId, url, navigationOptions = {}) { + return withLock(async () => { + if (automationOwnsTab(tabId)) return blockedResult(tabId); + return store.navigation(tabId, url, navigationOptions); + }); + }, + }; +} + +function sessionKey(tabId) { + const id = Number(tabId); + return Number.isFinite(id) ? `${TEACHER_SESSION_PREFIX}${id}` : ''; +} + +function normalizedScope(value) { + const origin = String(value?.origin || ''); + const pathFamily = String(value?.pathFamily || '/'); + const scope = workflowUrlScope(`${origin}${pathFamily}`); + return scope?.origin === origin ? scope : null; +} + +function normalizedSession(value, tabId) { + const key = sessionKey(tabId); + const name = normalizeSavedWorkflowName(value?.name); + const start = normalizedScope(value?.start); + const currentScope = normalizedScope(value?.currentScope) || start; + if (!key || value?.active !== true || !name || !start) return null; + const actions = []; + for (const raw of Array.isArray(value?.actions) ? value.actions : []) { + const action = normalizeTeacherCaptureAction(raw); + if (action) actions.push(action); + if (actions.length >= TEACHER_ACTION_LIMIT) break; + } + return { + version: 1, + active: true, + tabId: Number(tabId), + name, + start, + currentScope, + webbrainVersion: String(value.webbrainVersion || '').slice(0, 40), + startedAt: Math.max(1, Number(value.startedAt) || Date.now()), + updatedAt: Math.max(1, Number(value.updatedAt) || Date.now()), + actions, + lastActionAt: Math.max(0, Number(value.lastActionAt) || 0), + lastActionKind: String(value.lastActionKind || ''), + lastActionFingerprint: String(value.lastActionFingerprint || '').slice(0, 2000), + skippedActionCount: Math.max(0, Math.floor(Number(value.skippedActionCount) || 0)), + actionLimitReached: value.actionLimitReached === true, + }; +} + +export function createTeacherSessionStore(storageArea, options = {}) { + const now = options.now || Date.now; + const read = async (tabId) => { + const key = sessionKey(tabId); + if (!key) return null; + const stored = await storageArea.get(key); + return normalizedSession(stored?.[key], tabId); + }; + const write = async (session) => { + const normalized = normalizedSession(session, session?.tabId); + if (!normalized) throw new Error('Invalid teacher session.'); + normalized.updatedAt = now(); + await storageArea.set({ [sessionKey(normalized.tabId)]: normalized }); + return normalized; + }; + + const record = async (tabId, input) => { + const session = await read(tabId); + if (!session) return { changed: false, reason: 'no_active_session', session: null }; + const action = normalizeTeacherCaptureAction(input); + if (!action) { + const reason = ['click', 'field', 'checked'].includes(String(input?.kind || '')) + ? 'unsafe_target' + : 'invalid_action'; + session.skippedActionCount += 1; + return { changed: false, reason, session: await write(session) }; + } + const ts = now(); + const fingerprint = JSON.stringify(action); + if (action.kind === 'field' && action.submit === true && ts - session.lastActionAt < 2000) { + const lastIndex = session.actions.length - 1; + const lastAction = session.actions[lastIndex]; + const candidateIndex = lastAction?.kind === 'navigate' ? lastIndex - 1 : lastIndex; + const candidate = session.actions[candidateIndex]; + if ( + candidate?.kind === 'field' + && candidate.submit !== true + && JSON.stringify({ ...candidate, submit: true }) === fingerprint + ) { + session.actions[candidateIndex] = action; + if (candidateIndex !== lastIndex) session.actions.splice(candidateIndex + 1, 1); + if (candidateIndex === lastIndex) session.currentScope = action.scope; + session.lastActionAt = ts; + session.lastActionKind = action.kind; + session.lastActionFingerprint = fingerprint; + return { changed: true, reason: '', session: await write(session) }; + } + } + if (session.actions.length >= TEACHER_ACTION_LIMIT) { + if (!session.actionLimitReached) { + session.actionLimitReached = true; + await write(session); + } + return { changed: false, reason: 'action_limit', session }; + } + if (fingerprint === session.lastActionFingerprint && ts - session.lastActionAt < 200) { + return { changed: false, reason: 'duplicate', session }; + } + session.actions.push(action); + session.currentScope = action.kind === 'navigate' + ? workflowUrlScope(action.url) || session.currentScope + : action.scope; + session.lastActionAt = ts; + session.lastActionKind = action.kind; + session.lastActionFingerprint = fingerprint; + return { changed: true, reason: '', session: await write(session) }; + }; + + return { + async get(tabId) { return read(tabId); }, + async start(tabId, input = {}) { + const activeSession = await read(tabId); + if (activeSession) return { changed: false, reason: 'already_active', session: activeSession }; + const name = normalizeSavedWorkflowName(input.name); + if (!name) return { changed: false, reason: 'name_required', session: null }; + const start = workflowUrlScope(input.url); + if (!start) return { changed: false, reason: 'http_start_url_required', session: null }; + const ts = now(); + const session = await write({ + version: 1, + active: true, + tabId: Number(tabId), + name, + start, + currentScope: start, + webbrainVersion: input.webbrainVersion, + startedAt: ts, + updatedAt: ts, + actions: [], + }); + return { changed: true, reason: '', session }; + }, + record, + async navigation(tabId, url, options = {}) { + const session = await read(tabId); + if (!session) return { changed: false, reason: 'no_active_session', session: null }; + const destination = workflowUrlScope(url); + if (!destination) return { changed: false, reason: 'invalid_action', session }; + const previous = session.currentScope || session.start; + const sameScope = previous.origin === destination.origin + && previous.pathFamily === destination.pathFamily; + const lastAction = session.actions.at(-1); + const recentNavigationTrigger = options.force !== true + && ( + session.lastActionKind === 'click' + || (session.lastActionKind === 'field' && lastAction?.submit === true) + ) + && now() - session.lastActionAt < 2000; + session.currentScope = destination; + if (sameScope || recentNavigationTrigger) return { changed: true, reason: '', session: await write(session) }; + return record(tabId, { kind: 'navigate', scope: previous, url }); + }, + async clear(tabId) { + const key = sessionKey(tabId); + const session = await read(tabId); + if (!key || !session) return { changed: false, reason: 'no_active_session', session: null }; + await storageArea.remove(key); + return { changed: true, reason: '', session }; + }, + }; +} diff --git a/src/firefox/src/agent/workflows.js b/src/firefox/src/agent/workflows.js index aa323f4b1..775cd8fc2 100644 --- a/src/firefox/src/agent/workflows.js +++ b/src/firefox/src/agent/workflows.js @@ -391,6 +391,135 @@ export function compileWorkflowFromTrace(run, events, options = {}) { : { workflow: null, warnings, reason: 'normalization_failed' }; } +/** Normalize one value-free user-demonstrated action before session storage. */ +export function normalizeTeacherCaptureAction(input) { + const kind = cleanText(input?.kind, 40); + const scope = normalizeWorkflowScope(input?.scope) || workflowUrlScope(input?.pageUrl); + if (!scope) return null; + if (kind === 'navigate') { + const url = safeHttpUrl(input?.url); + return url ? { kind, scope, url } : null; + } + if (!['click', 'field', 'checked'].includes(kind)) return null; + const target = normalizeTarget(input?.target); + if (!isReplayableWorkflowTarget(target)) return null; + if (kind === 'checked' && typeof input?.checked !== 'boolean') return null; + return { + kind, + scope, + target, + ...(kind === 'checked' ? { checked: input.checked } : {}), + ...(kind === 'field' && input?.submit === true ? { submit: true } : {}), + }; +} + +/** Compile a user demonstration into the existing value-free workflow schema. */ +export function compileWorkflowFromDemonstration(input, options = {}) { + const ts = timestamp(options.now, nowMs()); + const name = normalizeSavedWorkflowName(input?.name); + if (!name) return { workflow: null, warnings: [], reason: 'name_required' }; + const start = normalizeWorkflowScope(input?.start) || workflowUrlScope(input?.startUrl); + if (!start) return { workflow: null, warnings: [], reason: 'http_start_url_required' }; + + const rawActions = Array.isArray(input?.actions) ? input.actions : []; + const previouslySkipped = Math.max(0, Math.floor(Number(input?.skippedActionCount) || 0)); + const warnings = previouslySkipped + ? [`Skipped ${previouslySkipped} demonstrated action(s) because no safe, reusable target was available.`] + : []; + if (input?.actionLimitReached === true) { + warnings.push(`Stopped recording additional actions after the ${MAX_STEPS}-step workflow limit.`); + } + const parameters = []; + const steps = []; + let skippedActionCount = previouslySkipped; + for (const raw of rawActions) { + const action = normalizeTeacherCaptureAction(raw); + if (!action) { + skippedActionCount += 1; + warnings.push('Skipped a demonstrated action because it had no safe, reusable target or URL.'); + continue; + } + if (action.kind === 'navigate') { + steps.push({ + id: `step_${steps.length + 1}`, + tool: 'navigate', + args: { url: action.url }, + scope: action.scope, + expected: { kind: 'url_changed' }, + }); + } else if (action.kind === 'click') { + steps.push({ + id: `step_${steps.length + 1}`, + tool: 'click_ax', + args: {}, + target: action.target, + scope: action.scope, + expected: { kind: 'tool_success' }, + }); + } else if (action.kind === 'checked') { + steps.push({ + id: `step_${steps.length + 1}`, + tool: 'set_checked', + args: { checked: action.checked }, + target: action.target, + scope: action.scope, + expected: { kind: 'checked', value: action.checked }, + }); + } else if (parameters.length < MAX_PARAMETERS) { + const base = parameterBase(action.target, `input_${parameters.length + 1}`); + const id = uniqueParameterId(base, parameters); + parameters.push({ + id, + label: cleanText( + action.target.label || action.target.ariaLabel || action.target.name + || action.target.fieldName || id, + 120, + ), + required: true, + sensitive: sensitiveParameter(action.target), + type: 'text', + }); + steps.push({ + id: `step_${steps.length + 1}`, + tool: action.submit ? 'set_field' : 'type_ax', + args: { + text: { [WORKFLOW_PARAM_REF_KEY]: id }, + clear: true, + ...(action.submit ? { submit: true } : {}), + }, + target: action.target, + scope: action.scope, + expected: { kind: 'tool_verified' }, + }); + } else { + skippedActionCount += 1; + warnings.push('Skipped a demonstrated field because the workflow parameter limit was reached.'); + } + if (steps.length >= MAX_STEPS) break; + } + + if (!steps.length) return { workflow: null, warnings, reason: 'no_replayable_steps' }; + const workflow = normalizeSavedWorkflow({ + schema: SAVED_WORKFLOW_SCHEMA, + id: options.id || createSavedWorkflowId(ts), + name, + createdAt: ts, + updatedAt: ts, + source: { runId: '', webbrainVersion: cleanText(input?.webbrainVersion, 40) }, + start, + parameters, + steps, + stats: { + sourceToolCount: rawActions.length + previouslySkipped, + compiledStepCount: steps.length, + skippedToolCount: skippedActionCount, + }, + }, { now: ts }); + return workflow + ? { workflow, warnings, reason: '' } + : { workflow: null, warnings, reason: 'normalization_failed' }; +} + function normalizeParameter(input) { const id = cleanId(input?.id).toLowerCase(); if (!id) return null; diff --git a/src/firefox/src/background.js b/src/firefox/src/background.js index a2cd4543f..725199a21 100644 --- a/src/firefox/src/background.js +++ b/src/firefox/src/background.js @@ -14,11 +14,13 @@ import { } from './agent/skills.js'; import { ScheduledJobManager } from './agent/scheduler.js'; import { + compileWorkflowFromDemonstration, compileLatestSuccessfulWorkflow, createSavedWorkflowStore, exportPortableWorkflowDefinition, importPortableWorkflowDefinition, } from './agent/workflows.js'; +import { createTeacherRunInterlock, createTeacherSessionStore } from './agent/teacher-mode.js'; import * as workflowTrace from './trace/recorder.js'; import { startClaudeOAuth, @@ -99,6 +101,13 @@ agent.setConversationScopeChangeListener((tabId, state) => { }); const userMemoryStore = createUserMemoryStore(browser.storage.local); const savedWorkflowStore = createSavedWorkflowStore(browser.storage.local); +const teacherSessionStore = createTeacherSessionStore(browser.storage.session); +const teacherRunInterlock = createTeacherRunInterlock(teacherSessionStore, { + automationOwnsTab: (tabId) => agent.isRunning(tabId) + || detachedRunStarts.has(tabId) + || scheduler.isRunning(tabId), +}); +agent.setRunStartGuard((tabId) => teacherRunInterlock.guardRunStart(tabId)); const profileSync = new ProfileSyncManager(browser.storage.local); const runCaptureController = createRunCaptureController({ api: browser, @@ -516,6 +525,33 @@ async function withSavedWorkflowStoreLock(task) { return run; } +async function withTeacherSessionStoreLock(task) { + return teacherRunInterlock.withLock(task); +} + +function publicTeacherSession(session) { + return session ? { + active: true, + name: session.name, + actionCount: session.actions?.length || 0, + startedAt: session.startedAt, + } : { active: false }; +} + +async function notifyTeacherState(tabId, session) { + if (tabId == null) return false; + try { + await browser.tabs.sendMessage(tabId, { + target: 'content', + action: 'teacher_state', + state: publicTeacherSession(session), + }); + return true; + } catch { + return false; + } +} + async function applyUserMemoryExtractionOperationsToCurrentStore(jobId, operations) { return withUserMemoryStoreLock(async () => { if (!await isUserMemoryExtractionEnabled()) { @@ -1186,6 +1222,7 @@ browser.tabs.onRemoved.addListener((tabId) => { contextMenuStorage.cleanup(tabId); tabChatHandoff.clear(tabId).catch(() => {}); scheduler.cancelForTab(tabId).catch(() => {}); + withTeacherSessionStoreLock(() => teacherSessionStore.clear(tabId)).catch(() => {}); try { agent._cleanupTab(tabId); } catch { /* ignore */ } }); @@ -1203,11 +1240,25 @@ function invalidateContextMenuForTab(tabId) { }).catch(() => {}); } +function recordTeacherNavigation(tabId, url, options) { + teacherRunInterlock.navigation(tabId, url, options).catch(() => {}); +} + +const TEACHER_EXPLICIT_NAVIGATION_TYPES = new Set([ + 'typed', 'auto_bookmark', 'generated', 'keyword', 'keyword_generated', +]); + browser.webNavigation?.onCommitted?.addListener?.((details) => { - if (details.frameId === 0) invalidateContextMenuForTab(details.tabId); + if (details.frameId !== 0) return; + recordTeacherNavigation(details.tabId, details.url, { + force: TEACHER_EXPLICIT_NAVIGATION_TYPES.has(details.transitionType), + }); + invalidateContextMenuForTab(details.tabId); }); browser.webNavigation?.onHistoryStateUpdated?.addListener?.((details) => { - if (details.frameId === 0) invalidateContextMenuForTab(details.tabId); + if (details.frameId !== 0) return; + recordTeacherNavigation(details.tabId, details.url); + invalidateContextMenuForTab(details.tabId); }); browser.webNavigation?.onReferenceFragmentUpdated?.addListener?.((details) => { if (details.frameId === 0) invalidateContextMenuForTab(details.tabId); @@ -1887,6 +1938,84 @@ async function handleMessage(msg, sender) { return { ok: true, ...result }; } + case 'get_teacher_mode': { + const tabId = msg.tabId || sender.tab?.id; + if (!tabId) return { ok: false, reason: 'tab_required', session: { active: false } }; + const session = await teacherSessionStore.get(tabId); + return { ok: true, session: publicTeacherSession(session) }; + } + + case 'start_teacher_mode': { + const tabId = msg.tabId || sender.tab?.id; + if (!tabId) return { ok: false, reason: 'tab_required' }; + if (agent.isRunning(tabId) || detachedRunStarts.has(tabId)) { + return { ok: false, reason: 'agent_running' }; + } + const tab = await browser.tabs.get(tabId).catch(() => null); + const result = await teacherRunInterlock.start(tabId, { + name: msg.name, + url: tab?.url, + webbrainVersion: browser.runtime.getManifest().version, + }); + if (result.changed && !await notifyTeacherState(tabId, result.session)) { + await withTeacherSessionStoreLock(() => teacherSessionStore.clear(tabId)); + return { ok: false, reason: 'capture_unavailable', session: { active: false } }; + } + return { ok: result.changed, reason: result.reason, session: publicTeacherSession(result.session) }; + } + + case 'record_teacher_action': { + const tabId = sender.tab?.id; + if (!tabId || (sender.frameId != null && sender.frameId !== 0)) { + return { ok: false, reason: 'tab_required' }; + } + const result = await teacherRunInterlock.record(tabId, msg.teacherAction); + return { + ok: result.changed || ['duplicate', 'unsafe_target'].includes(result.reason), + reason: result.reason, + session: publicTeacherSession(result.session), + }; + } + + case 'end_teacher_mode': { + const tabId = msg.tabId || sender.tab?.id; + if (!tabId) return { ok: false, reason: 'tab_required' }; + const response = await withTeacherSessionStoreLock(async () => { + let session = await teacherSessionStore.get(tabId); + if (!session) return { ok: false, reason: 'no_active_session', stopped: false }; + const flushed = await browser.tabs.sendMessage(tabId, { + target: 'content', + action: 'flush_teacher_capture', + }).catch(() => null); + if (flushed?.teacherAction) { + await teacherSessionStore.record(tabId, flushed.teacherAction); + session = await teacherSessionStore.get(tabId); + } + let result; + try { + const compiled = compileWorkflowFromDemonstration(session); + if (!compiled.workflow) { + result = { ok: false, ...compiled }; + } else { + const saved = await withSavedWorkflowStoreLock(() => savedWorkflowStore.put(compiled.workflow)); + result = { + ok: saved.changed, + workflow: saved.workflow, + warnings: compiled.warnings, + reason: saved.reason || '', + }; + } + } catch (error) { + result = { ok: false, reason: 'save_failed', error: error?.message || String(error) }; + } finally { + await teacherSessionStore.clear(tabId); + } + return { ...result, stopped: true }; + }); + if (response.stopped) await notifyTeacherState(tabId, null); + return response; + } + case 'list_saved_workflows': return { ok: true, workflows: await savedWorkflowStore.list() }; diff --git a/src/firefox/src/content/accessibility-tree.js b/src/firefox/src/content/accessibility-tree.js index 968ddf06c..44fd68ff4 100644 --- a/src/firefox/src/content/accessibility-tree.js +++ b/src/firefox/src/content/accessibility-tree.js @@ -1133,5 +1133,6 @@ window.__wb_ax_lookup = lookup; window.__wb_ax_ref = getOrMintRef; window.__wb_ax_name = getAccessibleName; + window.__wb_ax_role = getRole; window.__wb_ax_suggest = suggestNearRefs; })(); diff --git a/src/firefox/src/content/teacher-capture.js b/src/firefox/src/content/teacher-capture.js new file mode 100644 index 000000000..387b0c443 --- /dev/null +++ b/src/firefox/src/content/teacher-capture.js @@ -0,0 +1,251 @@ +/** Captures trusted, value-free user actions while /teach is active. */ +(() => { + if (window.top !== window || window.__webbrainTeacherCaptureInstalled) return; + window.__webbrainTeacherCaptureInstalled = true; + + const INDICATOR_ID = 'webbrain-teacher-indicator'; + const recordedFields = new WeakMap(); + const dirtyFields = new WeakSet(); + let active = false; + let sessionName = ''; + let pendingEnter = null; + + function clean(value, max = 240) { + return String(value || '').replace(/[\r\n]+/g, ' ').replace(/\s+/g, ' ').trim().slice(0, max); + } + + function safeHref(element) { + const raw = element?.getAttribute?.('href'); + if (!raw) return ''; + try { + const url = new URL(raw, location.href); + if (!['http:', 'https:'].includes(url.protocol)) return ''; + url.username = ''; + url.password = ''; + url.search = ''; + url.hash = ''; + return url.toString(); + } catch { + return ''; + } + } + + function inferredRole(element) { + const exported = window.__wb_ax_role?.(element); + if (exported) return clean(exported, 40); + const explicit = clean(element?.getAttribute?.('role'), 40); + if (explicit) return explicit; + const tag = element?.tagName?.toLowerCase?.() || ''; + if (tag === 'button') return 'button'; + if (tag === 'a') return 'link'; + if (tag === 'select') return 'combobox'; + if (tag === 'textarea') return 'textbox'; + if (tag === 'input') { + const type = clean(element.getAttribute('type'), 40).toLowerCase(); + if (type === 'checkbox') return 'checkbox'; + if (type === 'radio') return 'radio'; + if (['button', 'submit'].includes(type)) return 'button'; + return 'textbox'; + } + return 'generic'; + } + + function labelFor(element) { + const labels = Array.from(element?.labels || []) + .map((label) => clean(label.innerText || label.textContent, 120)) + .filter(Boolean); + if (labels.length) return labels.join(' ').slice(0, 120); + return clean(element?.closest?.('label')?.innerText, 120); + } + + function semanticTarget(element, { field = false } = {}) { + if (!(element instanceof Element)) return null; + const target = { + role: inferredRole(element), + name: field ? '' : clean(window.__wb_ax_name?.(element), 120), + label: field && element.matches('[contenteditable]') ? '' : labelFor(element), + id: clean(element.getAttribute('id'), 100), + fieldName: clean(element.getAttribute('name'), 100), + type: clean(element.getAttribute('type'), 40), + ariaLabel: clean(element.getAttribute('aria-label'), 120), + placeholder: clean(element.getAttribute('placeholder'), 120), + href: safeHref(element), + }; + return Object.fromEntries(Object.entries(target).filter(([, item]) => item)); + } + + function isTeacherUiEvent(event) { + return event.composedPath?.().some((node) => node?.id === INDICATOR_ID) === true; + } + + function actionableFromEvent(event) { + const source = event.composedPath?.().find((node) => node instanceof Element); + if (!source) return null; + const direct = source.closest?.([ + 'input', 'textarea', 'select', 'button', 'a[href]', 'label', 'summary', + '[role="button"]', '[role="link"]', '[role="checkbox"]', '[role="radio"]', + '[contenteditable]:not([contenteditable="false"])', '[onclick]', + ].join(',')); + if (direct?.tagName === 'LABEL' && direct.control) return direct.control; + return direct; + } + + function isCheckable(element) { + const type = clean(element?.getAttribute?.('type'), 40).toLowerCase(); + const role = clean(element?.getAttribute?.('role'), 40).toLowerCase(); + return ['checkbox', 'radio'].includes(type) || ['checkbox', 'radio'].includes(role); + } + + function isField(element) { + if (!(element instanceof Element)) return false; + if (element.matches('textarea,select,[contenteditable]:not([contenteditable="false"])')) return true; + if (!element.matches('input')) return false; + const type = clean(element.getAttribute('type') || 'text', 40).toLowerCase(); + return !['button', 'submit', 'reset', 'checkbox', 'radio', 'file', 'hidden', 'image'].includes(type); + } + + function isSubmitControl(element) { + if (!element?.matches?.('button,input')) return false; + const type = clean(element.getAttribute('type'), 40).toLowerCase(); + return element.matches('button') ? (!type || type === 'submit') : ['submit', 'image'].includes(type); + } + + function formFor(element) { + return element?.form || element?.closest?.('form') || null; + } + + function sendAction(action) { + if (!active || !action) return; + try { + chrome.runtime.sendMessage({ + target: 'background', + action: 'record_teacher_action', + teacherAction: { ...action, pageUrl: location.href }, + }).catch?.(() => {}); + } catch { /* extension context may be invalidated during navigation */ } + } + + function activeFieldAction() { + let element = document.activeElement; + while (element?.shadowRoot?.activeElement) element = element.shadowRoot.activeElement; + if (!dirtyFields.has(element) || !isField(element)) return null; + dirtyFields.delete(element); + return { kind: 'field', target: semanticTarget(element, { field: true }), pageUrl: location.href }; + } + + function updateIndicator() { + document.getElementById(INDICATOR_ID)?.remove(); + if (!active || !document.documentElement) return; + const indicator = document.createElement('div'); + indicator.id = INDICATOR_ID; + indicator.setAttribute('role', 'status'); + indicator.setAttribute('aria-live', 'polite'); + indicator.textContent = `● WebBrain Teach${sessionName ? ` · ${sessionName}` : ''}`; + Object.assign(indicator.style, { + position: 'fixed', left: '12px', bottom: '12px', zIndex: '2147483647', + padding: '7px 10px', borderRadius: '999px', background: '#241b3d', + color: '#f7f3ff', border: '1px solid #8b5cf6', font: '600 12px system-ui', + boxShadow: '0 4px 18px rgba(0,0,0,.3)', pointerEvents: 'none', + }); + document.documentElement.appendChild(indicator); + } + + function setState(state) { + active = state?.active === true; + sessionName = active ? clean(state?.name, 80) : ''; + updateIndicator(); + } + + document.addEventListener('click', (event) => { + if (!active || !event.isTrusted || isTeacherUiEvent(event)) return; + const element = actionableFromEvent(event); + if (!element) return; + const keyboardSubmitClick = ( + isSubmitControl(element) + && event.detail === 0 + && pendingEnter?.form + && formFor(element) === pendingEnter.form + && Date.now() - pendingEnter.at < 500 + ); + if (pendingEnter && !keyboardSubmitClick) pendingEnter = null; + if (isField(element) || keyboardSubmitClick) return; + if (isCheckable(element)) { + queueMicrotask(() => { + if (!active) return; + const checked = element.matches('input') + ? element.checked === true + : element.getAttribute('aria-checked') === 'true'; + sendAction({ kind: 'checked', checked, target: semanticTarget(element) }); + }); + return; + } + sendAction({ kind: 'click', target: semanticTarget(element) }); + }, true); + + document.addEventListener('input', (event) => { + if (!active || !event.isTrusted) return; + const element = actionableFromEvent(event); + if (isField(element)) dirtyFields.add(element); + }, true); + + document.addEventListener('change', (event) => { + if (!active || !event.isTrusted) return; + const element = actionableFromEvent(event); + if (!isField(element)) return; + if (Date.now() - (recordedFields.get(element) || 0) < 1000) return; + dirtyFields.delete(element); + sendAction({ kind: 'field', target: semanticTarget(element, { field: true }) }); + }, true); + + document.addEventListener('focusout', (event) => { + if (!active || !event.isTrusted) return; + const element = actionableFromEvent(event); + if (!element?.matches?.('[contenteditable]:not([contenteditable="false"])')) return; + if (!dirtyFields.has(element)) return; + dirtyFields.delete(element); + sendAction({ kind: 'field', target: semanticTarget(element, { field: true }) }); + }, true); + + document.addEventListener('keydown', (event) => { + if (!active || !event.isTrusted) return; + if (event.key !== 'Enter' || event.repeat) { + pendingEnter = null; + return; + } + const element = actionableFromEvent(event); + if (!isField(element) || element.matches('textarea,[contenteditable]')) return; + const action = { kind: 'field', target: semanticTarget(element, { field: true }) }; + const at = Date.now(); + dirtyFields.delete(element); + recordedFields.set(element, at); + pendingEnter = { element, form: formFor(element), action, at }; + sendAction(action); + }, true); + + document.addEventListener('submit', (event) => { + if (!active || !event.isTrusted) return; + const pending = pendingEnter; + if (!pending?.form || event.target !== pending.form || Date.now() - pending.at >= 1000) return; + // Wait until submit propagation finishes so page handlers have a chance to + // cancel autocomplete/custom Enter behavior before we mark it replayable. + queueMicrotask(() => { + if (!active || event.defaultPrevented || pendingEnter !== pending) return; + pendingEnter = null; + recordedFields.set(pending.element, Date.now()); + sendAction({ ...pending.action, submit: true }); + }); + }, true); + + chrome.runtime.onMessage.addListener((message, _sender, sendResponse) => { + if (message?.target !== 'content') return; + if (message.action === 'teacher_state') setState(message.state); + if (message.action === 'flush_teacher_capture') { + sendResponse({ teacherAction: active ? activeFieldAction() : null }); + } + }); + try { + chrome.runtime.sendMessage({ target: 'background', action: 'get_teacher_mode' }, (response) => { + if (!chrome.runtime.lastError) setState(response?.session); + }); + } catch { /* ignore unsupported pages */ } +})(); diff --git a/src/firefox/src/ui/locales/ar.js b/src/firefox/src/ui/locales/ar.js index acdf0618b..d9e800ba5 100644 --- a/src/firefox/src/ui/locales/ar.js +++ b/src/firefox/src/ui/locales/ar.js @@ -770,6 +770,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "لم يتم الحفظ: النص فارغ بعد التنظيف أو يبدو أنه يحتوي على سر (كلمة مرور أو مفتاح API أو رمز مميز).", "sp.memory.reason.not_found": "لا توجد ذاكرة محفوظة بهذا المعرّف.", "sp.slash.workflows": "عرض مهام سير العمل المحفوظة وإدارتها", + "sp.slash.teach": "سجّل إجراءاتك كمسار عمل محفوظ", "sp.slash.run_workflow": "تشغيل سير عمل محفوظ حسب المعرّف", "sp.workflows.run_prompt": "تشغيل سير العمل المحفوظ «{name}»", "sp.workflows.parameters_for": "أدخل معلمات «{name}». تُستخدم القيم لهذا التشغيل فقط.", @@ -785,6 +786,13 @@ export default { "sp.workflows.saved": "تم حفظ مهمة سير العمل «{name}».", "sp.workflows.deleted": "تم حذف مهمة سير العمل «{name}».", "sp.workflows.error": "خطأ في مهمة سير العمل: {msg}", + "sp.teach.started": "جارٍ تعليم “{name}”. نفّذ مسار العمل، ثم استخدم /teach --end.", + "sp.teach.status": "جارٍ تعليم “{name}”: تم التقاط {count} من الإجراءات.", + "sp.teach.inactive": "وضع التعليم غير نشط.", + "sp.teach.saved": "تم تعلم مسار العمل “{name}” في {count} من الخطوات.", + "sp.teach.already_active": "وضع التعليم نشط بالفعل في علامة التبويب هذه.", + "sp.teach.agent_running": "أوقف تشغيل الوكيل النشط قبل بدء وضع التعليم.", + "sp.teach.capture_unavailable": "لا يمكن لوضع التعليم التقاط هذه الصفحة. افتح أو أعد تحميل صفحة http(s) عادية وحاول مرة أخرى.", "sp.workflows.reason.no_trace": "لم يتم العثور على تشغيل ناجح ومسجل في هذه المحادثة. فعّل التتبع، وأكمل المهمة، ثم احفظها.", "sp.workflows.reason.no_steps": "لا يحتوي هذا التشغيل على إجراءات يمكن حفظها بأمان لإعادة التشغيل.", "sp.workflows.reason.not_found": "لا توجد مهمة سير عمل محفوظة بهذا المعرّف.", diff --git a/src/firefox/src/ui/locales/bn.js b/src/firefox/src/ui/locales/bn.js index 10866221d..48d14f71f 100644 --- a/src/firefox/src/ui/locales/bn.js +++ b/src/firefox/src/ui/locales/bn.js @@ -128,6 +128,7 @@ export default { 'sp.slash.show_memory': "সংরক্ষিত ব্যবহারকারী মেমরি দেখান", 'sp.slash.forget_memory': "আইডি দ্বারা একটি সংরক্ষিত মেমরি ভুলে যান", 'sp.slash.workflows': "সংরক্ষিত ওয়ার্কফ্লো তালিকাভুক্ত করুন এবং পরিচালনা করুন", + "sp.slash.teach": "আপনার কাজগুলো সংরক্ষিত ওয়ার্কফ্লো হিসেবে রেকর্ড করুন", 'sp.slash.run_workflow': "আইডি দ্বারা একটি সংরক্ষিত ওয়ার্কফ্লো চালান", 'sp.workflows.run_prompt': "সংরক্ষিত ওয়ার্কফ্লো চালান \"{name}\"", 'sp.workflows.parameters_for': "\"{name}\" এর জন্য পরামিতি লিখুন। মান শুধুমাত্র এই রানের জন্য ব্যবহার করা হয়.", @@ -373,6 +374,13 @@ export default { 'sp.workflows.saved': "সংরক্ষিত ওয়ার্কফ্লো \"{name}\"।", 'sp.workflows.deleted': "\"{name}\" কর্মপ্রবাহ মুছে ফেলা হয়েছে৷", 'sp.workflows.error': "কর্মপ্রবাহ ত্রুটি: {msg}", + "sp.teach.started": "“{name}” শেখানো হচ্ছে। ওয়ার্কফ্লোটি সম্পন্ন করে /teach --end ব্যবহার করুন।", + "sp.teach.status": "“{name}” শেখানো হচ্ছে: {count}টি কাজ ধরা হয়েছে।", + "sp.teach.inactive": "টিচার মোড সক্রিয় নয়।", + "sp.teach.saved": "“{name}” ওয়ার্কফ্লোটি {count}টি ধাপে শেখা হয়েছে।", + "sp.teach.already_active": "এই ট্যাবে টিচার মোড ইতিমধ্যে সক্রিয়।", + "sp.teach.agent_running": "টিচার মোড চালুর আগে সক্রিয় এজেন্ট রানটি বন্ধ করুন।", + "sp.teach.capture_unavailable": "টিচার মোড এই পৃষ্ঠাটি ধরতে পারছে না। একটি সাধারণ http(s) পৃষ্ঠা খুলুন বা রিলোড করে আবার চেষ্টা করুন।", 'sp.workflows.reason.no_trace': "এই কথোপকথনে কোনো সফল রেকর্ড করা রান পাওয়া যায়নি। ট্রেসিং সক্ষম করুন, কাজটি সম্পূর্ণ করুন, তারপরে এটি সংরক্ষণ করুন৷", 'sp.workflows.reason.no_steps': "সেই রানের কোনো অ্যাকশন নেই যা রিপ্লে করার জন্য নিরাপদে সংরক্ষণ করা যায়।", 'sp.workflows.reason.not_found': "কোনও সংরক্ষিত ওয়ার্কফ্লোতে সেই আইডি নেই।", diff --git a/src/firefox/src/ui/locales/de.js b/src/firefox/src/ui/locales/de.js index 1dcdea393..e2e8575e6 100644 --- a/src/firefox/src/ui/locales/de.js +++ b/src/firefox/src/ui/locales/de.js @@ -124,6 +124,7 @@ export default { 'sp.slash.show_memory': 'Gespeicherte Benutzermemorie anzeigen', 'sp.slash.forget_memory': 'Gespeicherte Memorie nach ID vergessen', 'sp.slash.workflows': 'Gespeicherte Workflows auflisten und verwalten', + "sp.slash.teach": "Aktionen als gespeicherten Workflow aufzeichnen", 'sp.slash.run_workflow': 'Gespeicherten Workflow nach ID ausführen', 'sp.slash.save_workflow': 'Letzten erfolgreichen aufgezeichneten Durchlauf speichern', 'sp.slash.delete_workflow': 'Gespeicherten Workflow nach ID löschen', @@ -363,6 +364,13 @@ export default { 'sp.workflows.saved': 'Workflow „{name}" gespeichert.', 'sp.workflows.deleted': 'Workflow „{name}" gelöscht.', 'sp.workflows.error': 'Workflow-Fehler: {msg}', + "sp.teach.started": "„{name}“ wird gelernt. Führe den Workflow aus und verwende dann /teach --end.", + "sp.teach.status": "„{name}“ wird gelernt: {count} Aktionen erfasst.", + "sp.teach.inactive": "Der Lernmodus ist nicht aktiv.", + "sp.teach.saved": "Workflow „{name}“ mit {count} Schritten gelernt.", + "sp.teach.already_active": "Der Lernmodus ist in diesem Tab bereits aktiv.", + "sp.teach.agent_running": "Beende den aktiven Agent-Lauf, bevor du den Lernmodus startest.", + "sp.teach.capture_unavailable": "Der Lernmodus kann diese Seite nicht erfassen. Öffne oder lade eine normale http(s)-Seite neu und versuche es erneut.", 'sp.workflows.reason.no_trace': 'In diesem Gespräch wurde kein erfolgreicher aufgezeichneter Durchlauf gefunden. Aktivieren Sie die Verfolgung, schließen Sie die Aufgabe ab und speichern Sie sie dann.', 'sp.workflows.reason.no_steps': 'Dieser Durchlauf enthält keine Aktionen, die sicher für die Wiedergabe gespeichert werden können.', 'sp.workflows.reason.not_found': 'Kein gespeicherter Workflow mit dieser ID.', diff --git a/src/firefox/src/ui/locales/en.js b/src/firefox/src/ui/locales/en.js index efac5f902..d63291e0d 100644 --- a/src/firefox/src/ui/locales/en.js +++ b/src/firefox/src/ui/locales/en.js @@ -128,6 +128,7 @@ export default { 'sp.slash.show_memory': 'Show saved user memory', 'sp.slash.forget_memory': 'Forget a saved memory by ID', 'sp.slash.workflows': 'List and manage saved workflows', + "sp.slash.teach": "Record your actions as a saved workflow", 'sp.slash.run_workflow': 'Run a saved workflow by ID', 'sp.workflows.run_prompt': 'Run saved workflow “{name}”', 'sp.workflows.parameters_for': 'Enter parameters for “{name}”. Values are used only for this run.', @@ -373,6 +374,13 @@ export default { 'sp.workflows.saved': 'Saved workflow “{name}”.', 'sp.workflows.deleted': 'Deleted workflow “{name}”.', 'sp.workflows.error': 'Workflow error: {msg}', + "sp.teach.started": "Teaching “{name}”. Perform the workflow, then use /teach --end.", + "sp.teach.status": "Teaching “{name}”: {count} actions captured.", + "sp.teach.inactive": "Teacher mode is not active.", + "sp.teach.saved": "Learned workflow “{name}” with {count} steps.", + "sp.teach.already_active": "Teacher mode is already active in this tab.", + "sp.teach.agent_running": "Stop the active agent run before starting Teacher mode.", + "sp.teach.capture_unavailable": "Teacher mode cannot capture this page. Open or reload a regular http(s) page and try again.", 'sp.workflows.reason.no_trace': 'No successful recorded run was found in this conversation. Enable tracing, complete the task, then save it.', 'sp.workflows.reason.no_steps': 'That run has no actions that can be safely saved for replay.', 'sp.workflows.reason.not_found': 'No saved workflow has that ID.', diff --git a/src/firefox/src/ui/locales/es.js b/src/firefox/src/ui/locales/es.js index c8b6f9d2d..bab02fb56 100644 --- a/src/firefox/src/ui/locales/es.js +++ b/src/firefox/src/ui/locales/es.js @@ -770,6 +770,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "No se guardó: el texto queda vacío tras la limpieza o parece contener un secreto (contraseña, clave de API o token).", "sp.memory.reason.not_found": "Ninguna memoria guardada tiene ese ID.", "sp.slash.workflows": "Listar y administrar flujos de trabajo guardados", + "sp.slash.teach": "Graba tus acciones como un flujo de trabajo guardado", "sp.slash.run_workflow": "Ejecutar un flujo guardado por ID", "sp.workflows.run_prompt": "Ejecutar el flujo guardado «{name}»", "sp.workflows.parameters_for": "Introduce los parámetros de «{name}». Los valores solo se usan en esta ejecución.", @@ -785,6 +786,13 @@ export default { "sp.workflows.saved": "Flujo de trabajo «{name}» guardado.", "sp.workflows.deleted": "Flujo de trabajo «{name}» eliminado.", "sp.workflows.error": "Error del flujo de trabajo: {msg}", + "sp.teach.started": "Enseñando “{name}”. Completa el flujo y usa /teach --end.", + "sp.teach.status": "Enseñando “{name}”: {count} acciones capturadas.", + "sp.teach.inactive": "El modo de enseñanza no está activo.", + "sp.teach.saved": "Se aprendió el flujo “{name}” con {count} pasos.", + "sp.teach.already_active": "El modo de enseñanza ya está activo en esta pestaña.", + "sp.teach.agent_running": "Detén la ejecución activa del agente antes de iniciar el modo de enseñanza.", + "sp.teach.capture_unavailable": "El modo de enseñanza no puede capturar esta página. Abre o recarga una página http(s) normal e inténtalo de nuevo.", "sp.workflows.reason.no_trace": "No se encontró una ejecución registrada y correcta en esta conversación. Activa el seguimiento, completa la tarea y guárdala.", "sp.workflows.reason.no_steps": "Esa ejecución no contiene acciones que puedan guardarse de forma segura.", "sp.workflows.reason.not_found": "No existe un flujo de trabajo guardado con ese ID.", diff --git a/src/firefox/src/ui/locales/fa.js b/src/firefox/src/ui/locales/fa.js index dcfc2c07f..fa13f0e6e 100644 --- a/src/firefox/src/ui/locales/fa.js +++ b/src/firefox/src/ui/locales/fa.js @@ -128,6 +128,7 @@ export default { 'sp.slash.show_memory': "نمایش حافظه کاربر ذخیره شده", 'sp.slash.forget_memory': "یک حافظه ذخیره شده توسط ID را فراموش کنید", 'sp.slash.workflows': "گردش کار ذخیره شده را فهرست و مدیریت کنید", + "sp.slash.teach": "اقدامات خود را به‌صورت گردش‌کار ذخیره‌شده ضبط کنید", 'sp.slash.run_workflow': "یک گردش کار ذخیره شده توسط ID را اجرا کنید", 'sp.workflows.run_prompt': "اجرای گردش کار ذخیره شده \"{name}\"", 'sp.workflows.parameters_for': "پارامترهای \"{name}\" را وارد کنید. مقادیر فقط برای این اجرا استفاده می شود.", @@ -373,6 +374,13 @@ export default { 'sp.workflows.saved': "گردش کار ذخیره شده \"{name}\".", 'sp.workflows.deleted': "گردش کار حذف شده \"{name}\".", 'sp.workflows.error': "خطای گردش کار: {msg}", + "sp.teach.started": "در حال آموزش «{name}». گردش‌کار را انجام دهید، سپس /teach --end را وارد کنید.", + "sp.teach.status": "در حال آموزش «{name}»: {count} اقدام ثبت شد.", + "sp.teach.inactive": "حالت آموزش فعال نیست.", + "sp.teach.saved": "گردش‌کار «{name}» با {count} مرحله آموخته شد.", + "sp.teach.already_active": "حالت آموزش از قبل در این برگه فعال است.", + "sp.teach.agent_running": "پیش از شروع حالت آموزش، اجرای فعال عامل را متوقف کنید.", + "sp.teach.capture_unavailable": "حالت آموزش نمی‌تواند این صفحه را ثبت کند. یک صفحه معمولی http(s) را باز یا بازخوانی کنید و دوباره تلاش کنید.", 'sp.workflows.reason.no_trace': "هیچ اجرا ضبط شده موفقی در این مکالمه یافت نشد. ردیابی را فعال کنید، کار را کامل کنید، سپس آن را ذخیره کنید.", 'sp.workflows.reason.no_steps': "این اجرا هیچ عملکردی ندارد که بتوان با خیال راحت برای پخش مجدد ذخیره کرد.", 'sp.workflows.reason.not_found': "هیچ گردش کار ذخیره شده ای آن شناسه را ندارد.", diff --git a/src/firefox/src/ui/locales/fr.js b/src/firefox/src/ui/locales/fr.js index 8c0432a1a..a76d2e9af 100644 --- a/src/firefox/src/ui/locales/fr.js +++ b/src/firefox/src/ui/locales/fr.js @@ -770,6 +770,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "Non enregistré : le texte est vide après nettoyage ou semble contenir un secret (mot de passe, clé API, jeton).", "sp.memory.reason.not_found": "Aucune mémoire enregistrée ne possède cet identifiant.", "sp.slash.workflows": "Lister et gérer les workflows enregistrés", + "sp.slash.teach": "Enregistrer vos actions comme workflow sauvegardé", "sp.slash.run_workflow": "Exécuter un workflow enregistré par ID", "sp.workflows.run_prompt": "Exécuter le workflow enregistré « {name} »", "sp.workflows.parameters_for": "Saisissez les paramètres de « {name} ». Les valeurs ne servent qu’à cette exécution.", @@ -785,6 +786,13 @@ export default { "sp.workflows.saved": "Workflow « {name} » enregistré.", "sp.workflows.deleted": "Workflow « {name} » supprimé.", "sp.workflows.error": "Erreur de workflow : {msg}", + "sp.teach.started": "Apprentissage de « {name} ». Effectuez le workflow, puis utilisez /teach --end.", + "sp.teach.status": "Apprentissage de « {name} » : {count} actions capturées.", + "sp.teach.inactive": "Le mode d’apprentissage n’est pas actif.", + "sp.teach.saved": "Workflow « {name} » appris avec {count} étapes.", + "sp.teach.already_active": "Le mode d’apprentissage est déjà actif dans cet onglet.", + "sp.teach.agent_running": "Arrêtez l’exécution active de l’agent avant de lancer le mode d’apprentissage.", + "sp.teach.capture_unavailable": "Le mode d’apprentissage ne peut pas capturer cette page. Ouvrez ou rechargez une page http(s) normale, puis réessayez.", "sp.workflows.reason.no_trace": "Aucune exécution réussie et tracée dans cette conversation. Activez le traçage, terminez la tâche, puis enregistrez-la.", "sp.workflows.reason.no_steps": "Cette exécution ne contient aucune action pouvant être enregistrée de manière sûre.", "sp.workflows.reason.not_found": "Aucun workflow enregistré ne possède cet ID.", diff --git a/src/firefox/src/ui/locales/he.js b/src/firefox/src/ui/locales/he.js index 946ca35bd..6e14260bc 100644 --- a/src/firefox/src/ui/locales/he.js +++ b/src/firefox/src/ui/locales/he.js @@ -413,6 +413,7 @@ export default { "st.tab.vision": "ראייה", "st.tab.multimodal": "מולטי-מודאלי", "sp.slash.workflows": "הצגה וניהול של תהליכי עבודה שמורים", + "sp.slash.teach": "הקלטת הפעולות שלך כתהליך עבודה שמור", "sp.slash.run_workflow": "הפעלת תהליך עבודה שמור לפי מזהה", "sp.workflows.run_prompt": "הפעלת תהליך העבודה השמור „{name}”", "sp.workflows.parameters_for": "יש להזין פרמטרים עבור „{name}”. הערכים משמשים להרצה זו בלבד.", @@ -428,6 +429,13 @@ export default { "sp.workflows.saved": "תהליך העבודה „{name}” נשמר.", "sp.workflows.deleted": "תהליך העבודה „{name}” נמחק.", "sp.workflows.error": "שגיאת תהליך עבודה: {msg}", + "sp.teach.started": "מלמד את “{name}”. בצע את התהליך ואז השתמש ב־/teach --end.", + "sp.teach.status": "מלמד את “{name}”: נקלטו {count} פעולות.", + "sp.teach.inactive": "מצב לימוד אינו פעיל.", + "sp.teach.saved": "תהליך העבודה “{name}” נלמד עם {count} שלבים.", + "sp.teach.already_active": "מצב לימוד כבר פעיל בכרטיסייה זו.", + "sp.teach.agent_running": "יש לעצור את הרצת הסוכן הפעילה לפני הפעלת מצב לימוד.", + "sp.teach.capture_unavailable": "מצב לימוד אינו יכול לקלוט דף זה. פתח או טען מחדש דף http(s) רגיל ונסה שוב.", "sp.workflows.reason.no_trace": "לא נמצאה בשיחה זו הרצה מוצלחת ומוקלטת. הפעילו מעקב, השלימו את המשימה ושמרו אותה.", "sp.workflows.reason.no_steps": "בהרצה זו אין פעולות שניתן לשמור בבטחה להפעלה חוזרת.", "sp.workflows.reason.not_found": "אין תהליך עבודה שמור עם מזהה זה.", diff --git a/src/firefox/src/ui/locales/hi.js b/src/firefox/src/ui/locales/hi.js index 01972d9a0..b72da3273 100644 --- a/src/firefox/src/ui/locales/hi.js +++ b/src/firefox/src/ui/locales/hi.js @@ -128,6 +128,7 @@ export default { 'sp.slash.show_memory': "सहेजी गई उपयोगकर्ता मेमोरी दिखाएँ", 'sp.slash.forget_memory': "आईडी द्वारा सहेजी गई मेमोरी को भूल जाएं", 'sp.slash.workflows': "सहेजे गए वर्कफ़्लो को सूचीबद्ध करें और प्रबंधित करें", + "sp.slash.teach": "अपने कार्यों को सहेजे गए वर्कफ़्लो के रूप में रिकॉर्ड करें", 'sp.slash.run_workflow': "आईडी द्वारा सहेजा गया वर्कफ़्लो चलाएँ", 'sp.workflows.run_prompt': "सहेजे गए वर्कफ़्लो \"{name}\" चलाएँ", 'sp.workflows.parameters_for': "\"{name}\" के लिए पैरामीटर दर्ज करें। मानों का उपयोग केवल इस रन के लिए किया जाता है।", @@ -373,6 +374,13 @@ export default { 'sp.workflows.saved': "सहेजा गया वर्कफ़्लो \"{name}\"।", 'sp.workflows.deleted': "हटाया गया वर्कफ़्लो \"{name}\"।", 'sp.workflows.error': "वर्कफ़्लो त्रुटि: {msg}", + "sp.teach.started": "“{name}” सिखाया जा रहा है। वर्कफ़्लो पूरा करें, फिर /teach --end चलाएँ।", + "sp.teach.status": "“{name}” सिखाया जा रहा है: {count} कार्य कैप्चर हुए।", + "sp.teach.inactive": "टीचर मोड सक्रिय नहीं है।", + "sp.teach.saved": "“{name}” वर्कफ़्लो {count} चरणों के साथ सीख लिया गया।", + "sp.teach.already_active": "इस टैब में टीचर मोड पहले से सक्रिय है।", + "sp.teach.agent_running": "टीचर मोड शुरू करने से पहले सक्रिय एजेंट रन रोकें।", + "sp.teach.capture_unavailable": "टीचर मोड इस पेज को कैप्चर नहीं कर सकता। किसी सामान्य http(s) पेज को खोलें या रीलोड करें और फिर कोशिश करें।", 'sp.workflows.reason.no_trace': "इस बातचीत में कोई सफल रिकॉर्डेड रन नहीं मिला. ट्रेसिंग सक्षम करें, कार्य पूरा करें, फिर इसे सहेजें।", 'sp.workflows.reason.no_steps': "उस रन में ऐसी कोई क्रिया नहीं है जिसे पुनः चलाने के लिए सुरक्षित रूप से सहेजा जा सके।", 'sp.workflows.reason.not_found': "किसी भी सहेजे गए वर्कफ़्लो में वह आईडी नहीं है।", diff --git a/src/firefox/src/ui/locales/id.js b/src/firefox/src/ui/locales/id.js index 8fb01f083..278e0fc36 100644 --- a/src/firefox/src/ui/locales/id.js +++ b/src/firefox/src/ui/locales/id.js @@ -770,6 +770,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "Tidak disimpan: teks kosong setelah dibersihkan atau tampaknya berisi rahasia (kata sandi, kunci API, token).", "sp.memory.reason.not_found": "Tidak ada memori tersimpan dengan ID tersebut.", "sp.slash.workflows": "Lihat dan kelola alur kerja tersimpan", + "sp.slash.teach": "Rekam tindakan Anda sebagai alur kerja tersimpan", "sp.slash.run_workflow": "Jalankan alur kerja tersimpan berdasarkan ID", "sp.workflows.run_prompt": "Jalankan alur kerja tersimpan “{name}”", "sp.workflows.parameters_for": "Masukkan parameter untuk “{name}”. Nilai hanya digunakan untuk proses ini.", @@ -785,6 +786,13 @@ export default { "sp.workflows.saved": "Alur kerja “{name}” disimpan.", "sp.workflows.deleted": "Alur kerja “{name}” dihapus.", "sp.workflows.error": "Kesalahan alur kerja: {msg}", + "sp.teach.started": "Mengajarkan “{name}”. Jalankan alur kerja, lalu gunakan /teach --end.", + "sp.teach.status": "Mengajarkan “{name}”: {count} tindakan direkam.", + "sp.teach.inactive": "Mode pengajaran tidak aktif.", + "sp.teach.saved": "Alur kerja “{name}” dipelajari dengan {count} langkah.", + "sp.teach.already_active": "Mode pengajaran sudah aktif di tab ini.", + "sp.teach.agent_running": "Hentikan agen yang sedang berjalan sebelum memulai mode pengajaran.", + "sp.teach.capture_unavailable": "Mode pengajaran tidak dapat merekam halaman ini. Buka atau muat ulang halaman http(s) biasa lalu coba lagi.", "sp.workflows.reason.no_trace": "Tidak ada proses terekam yang berhasil dalam percakapan ini. Aktifkan pelacakan, selesaikan tugas, lalu simpan.", "sp.workflows.reason.no_steps": "Proses tersebut tidak memiliki tindakan yang aman untuk disimpan dan dijalankan ulang.", "sp.workflows.reason.not_found": "Tidak ada alur kerja tersimpan dengan ID tersebut.", diff --git a/src/firefox/src/ui/locales/ja.js b/src/firefox/src/ui/locales/ja.js index ebb62c915..63292d073 100644 --- a/src/firefox/src/ui/locales/ja.js +++ b/src/firefox/src/ui/locales/ja.js @@ -770,6 +770,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "保存されませんでした。整理後のテキストが空か、秘密情報（パスワード、API キー、トークン）が含まれているようです。", "sp.memory.reason.not_found": "その ID の保存済みメモリはありません。", "sp.slash.workflows": "保存済みワークフローを一覧・管理", + "sp.slash.teach": "操作を保存済みワークフローとして記録", "sp.slash.run_workflow": "保存済みワークフローを ID で実行", "sp.workflows.run_prompt": "保存済みワークフロー「{name}」を実行", "sp.workflows.parameters_for": "「{name}」のパラメーターを入力してください。値は今回の実行だけに使用されます。", @@ -785,6 +786,13 @@ export default { "sp.workflows.saved": "ワークフロー「{name}」を保存しました。", "sp.workflows.deleted": "ワークフロー「{name}」を削除しました。", "sp.workflows.error": "ワークフローエラー: {msg}", + "sp.teach.started": "「{name}」を学習中です。操作を完了してから /teach --end を使用してください。", + "sp.teach.status": "「{name}」を学習中：{count} 件の操作を記録しました。", + "sp.teach.inactive": "ティーチャーモードは有効ではありません。", + "sp.teach.saved": "ワークフロー「{name}」を {count} ステップで学習しました。", + "sp.teach.already_active": "このタブではティーチャーモードがすでに有効です。", + "sp.teach.agent_running": "ティーチャーモードを開始する前に実行中のエージェントを停止してください。", + "sp.teach.capture_unavailable": "このページはティーチャーモードで記録できません。通常の http(s) ページを開くか再読み込みして、もう一度お試しください。", "sp.workflows.reason.no_trace": "この会話に成功した記録済み実行がありません。トレースを有効にしてタスクを完了し、保存してください。", "sp.workflows.reason.no_steps": "この実行には安全に保存して再実行できる操作がありません。", "sp.workflows.reason.not_found": "そのIDの保存済みワークフローはありません。", diff --git a/src/firefox/src/ui/locales/ko.js b/src/firefox/src/ui/locales/ko.js index e06872215..faff9fad1 100644 --- a/src/firefox/src/ui/locales/ko.js +++ b/src/firefox/src/ui/locales/ko.js @@ -770,6 +770,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "저장하지 않았습니다. 정리 후 텍스트가 비어 있거나 비밀 정보(비밀번호, API 키, 토큰)가 포함된 것으로 보입니다.", "sp.memory.reason.not_found": "해당 ID의 저장된 메모리가 없습니다.", "sp.slash.workflows": "저장된 워크플로 목록 및 관리", + "sp.slash.teach": "작업을 저장된 워크플로로 기록", "sp.slash.run_workflow": "ID로 저장된 워크플로 실행", "sp.workflows.run_prompt": "저장된 워크플로 “{name}” 실행", "sp.workflows.parameters_for": "“{name}”의 매개변수를 입력하세요. 값은 이번 실행에만 사용됩니다.", @@ -785,6 +786,13 @@ export default { "sp.workflows.saved": "워크플로 “{name}”을 저장했습니다.", "sp.workflows.deleted": "워크플로 “{name}”을 삭제했습니다.", "sp.workflows.error": "워크플로 오류: {msg}", + "sp.teach.started": "“{name}”을 학습 중입니다. 작업을 완료한 다음 /teach --end를 사용하세요.", + "sp.teach.status": "“{name}” 학습 중: {count}개 작업을 캡처했습니다.", + "sp.teach.inactive": "교사 모드가 활성화되어 있지 않습니다.", + "sp.teach.saved": "“{name}” 워크플로를 {count}단계로 학습했습니다.", + "sp.teach.already_active": "이 탭에서 교사 모드가 이미 활성화되어 있습니다.", + "sp.teach.agent_running": "교사 모드를 시작하기 전에 실행 중인 에이전트를 중지하세요.", + "sp.teach.capture_unavailable": "교사 모드에서 이 페이지를 캡처할 수 없습니다. 일반 http(s) 페이지를 열거나 새로고침한 후 다시 시도하세요.", "sp.workflows.reason.no_trace": "이 대화에서 성공한 기록 실행을 찾지 못했습니다. 추적을 켜고 작업을 완료한 다음 저장하세요.", "sp.workflows.reason.no_steps": "이 실행에는 안전하게 저장하여 다시 실행할 수 있는 동작이 없습니다.", "sp.workflows.reason.not_found": "해당 ID의 저장된 워크플로가 없습니다.", diff --git a/src/firefox/src/ui/locales/ms.js b/src/firefox/src/ui/locales/ms.js index b726e71ae..015b88d58 100644 --- a/src/firefox/src/ui/locales/ms.js +++ b/src/firefox/src/ui/locales/ms.js @@ -770,6 +770,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "Tidak disimpan: teks kosong selepas dibersihkan atau kelihatan mengandungi rahsia (kata laluan, kunci API, token).", "sp.memory.reason.not_found": "Tiada memori yang disimpan dengan ID tersebut.", "sp.slash.workflows": "Senarai dan urus aliran kerja tersimpan", + "sp.slash.teach": "Rakam tindakan anda sebagai aliran kerja tersimpan", "sp.slash.run_workflow": "Jalankan aliran kerja tersimpan mengikut ID", "sp.workflows.run_prompt": "Jalankan aliran kerja tersimpan “{name}”", "sp.workflows.parameters_for": "Masukkan parameter untuk “{name}”. Nilai hanya digunakan untuk larian ini.", @@ -785,6 +786,13 @@ export default { "sp.workflows.saved": "Aliran kerja “{name}” disimpan.", "sp.workflows.deleted": "Aliran kerja “{name}” dipadam.", "sp.workflows.error": "Ralat aliran kerja: {msg}", + "sp.teach.started": "Mengajar “{name}”. Lakukan aliran kerja, kemudian gunakan /teach --end.", + "sp.teach.status": "Mengajar “{name}”: {count} tindakan dirakam.", + "sp.teach.inactive": "Mod pengajaran tidak aktif.", + "sp.teach.saved": "Aliran kerja “{name}” dipelajari dengan {count} langkah.", + "sp.teach.already_active": "Mod pengajaran sudah aktif dalam tab ini.", + "sp.teach.agent_running": "Hentikan ejen yang sedang berjalan sebelum memulakan mod pengajaran.", + "sp.teach.capture_unavailable": "Mod pengajaran tidak dapat merakam halaman ini. Buka atau muat semula halaman http(s) biasa dan cuba lagi.", "sp.workflows.reason.no_trace": "Tiada pelaksanaan berjaya yang direkodkan dalam perbualan ini. Aktifkan penjejakan, lengkapkan tugas, kemudian simpannya.", "sp.workflows.reason.no_steps": "Pelaksanaan itu tiada tindakan yang boleh disimpan dengan selamat untuk dimainkan semula.", "sp.workflows.reason.not_found": "Tiada aliran kerja tersimpan dengan ID tersebut.", diff --git a/src/firefox/src/ui/locales/nl.js b/src/firefox/src/ui/locales/nl.js index 544af24ed..c22dda16e 100644 --- a/src/firefox/src/ui/locales/nl.js +++ b/src/firefox/src/ui/locales/nl.js @@ -124,6 +124,7 @@ export default { 'sp.slash.show_memory': 'Toon opgeslagen gebruikersgeheugen', 'sp.slash.forget_memory': 'Een opgeslagen geheugen vergeten op ID', 'sp.slash.workflows': 'Lijst en beheer opgeslagen workflows', + "sp.slash.teach": "Acties opnemen als opgeslagen workflow", 'sp.slash.run_workflow': 'Voer een opgeslagen workflow uit op ID', 'sp.workflows.run_prompt': 'Opgeslagen workflow "{name}" uitvoeren', 'sp.workflows.parameters_for': 'Voer parameters in voor "{name}". Waarden worden alleen voor deze uitvoering gebruikt.', @@ -359,6 +360,13 @@ export default { 'sp.workflows.saved': 'Workflow "{name}" opgeslagen.', 'sp.workflows.deleted': 'Workflow "{name}" verwijderd.', 'sp.workflows.error': 'Workflowfout: {msg}', + "sp.teach.started": "“{name}” wordt aangeleerd. Voer de workflow uit en gebruik daarna /teach --end.", + "sp.teach.status": "“{name}” wordt aangeleerd: {count} acties vastgelegd.", + "sp.teach.inactive": "De leermodus is niet actief.", + "sp.teach.saved": "Workflow “{name}” met {count} stappen aangeleerd.", + "sp.teach.already_active": "De leermodus is al actief in dit tabblad.", + "sp.teach.agent_running": "Stop de actieve agenttaak voordat je de leermodus start.", + "sp.teach.capture_unavailable": "De leermodus kan deze pagina niet vastleggen. Open of herlaad een gewone http(s)-pagina en probeer opnieuw.", 'sp.workflows.reason.no_trace': 'Er is geen succesvolle opgenomen uitvoering gevonden in dit gesprek...', 'sp.workflows.reason.no_steps': 'Die uitvoering heeft geen acties die veilig kunnen worden opgeslagen voor herhaling.', 'sp.workflows.reason.not_found': 'Geen opgeslagen workflow heeft die ID.', diff --git a/src/firefox/src/ui/locales/pl.js b/src/firefox/src/ui/locales/pl.js index f2eb75c79..4f548c001 100644 --- a/src/firefox/src/ui/locales/pl.js +++ b/src/firefox/src/ui/locales/pl.js @@ -761,6 +761,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "Nie zapisano: po oczyszczeniu tekst jest pusty lub wygląda na zawierający sekret (hasło, klucz API, token).", "sp.memory.reason.not_found": "Nie ma zapisanego wpisu o tym ID.", "sp.slash.workflows": "Wyświetlaj i zarządzaj zapisanymi przepływami pracy", + "sp.slash.teach": "Nagraj działania jako zapisany przepływ pracy", "sp.slash.run_workflow": "Uruchom zapisany przepływ pracy według ID", "sp.workflows.run_prompt": "Uruchom zapisany przepływ pracy „{name}”", "sp.workflows.parameters_for": "Wprowadź parametry dla „{name}”. Wartości zostaną użyte tylko w tym uruchomieniu.", @@ -776,6 +777,13 @@ export default { "sp.workflows.saved": "Zapisano przepływ „{name}”.", "sp.workflows.deleted": "Usunięto przepływ „{name}”.", "sp.workflows.error": "Błąd przepływu pracy: {msg}", + "sp.teach.started": "Uczenie „{name}”. Wykonaj przepływ pracy, a następnie użyj /teach --end.", + "sp.teach.status": "Uczenie „{name}”: zarejestrowano {count} działań.", + "sp.teach.inactive": "Tryb nauczania nie jest aktywny.", + "sp.teach.saved": "Nauczono przepływ „{name}” z {count} krokami.", + "sp.teach.already_active": "Tryb nauczania jest już aktywny w tej karcie.", + "sp.teach.agent_running": "Zatrzymaj aktywne działanie agenta przed uruchomieniem trybu nauczania.", + "sp.teach.capture_unavailable": "Tryb nauczania nie może rejestrować tej strony. Otwórz lub odśwież zwykłą stronę http(s) i spróbuj ponownie.", "sp.workflows.reason.no_trace": "W tej rozmowie nie znaleziono udanego zarejestrowanego wykonania. Włącz śledzenie, ukończ zadanie i zapisz je.", "sp.workflows.reason.no_steps": "To wykonanie nie zawiera działań, które można bezpiecznie zapisać do ponownego użycia.", "sp.workflows.reason.not_found": "Nie ma zapisanego przepływu pracy o tym ID.", diff --git a/src/firefox/src/ui/locales/pt.js b/src/firefox/src/ui/locales/pt.js index 3146b2c7a..93d3f9b3a 100644 --- a/src/firefox/src/ui/locales/pt.js +++ b/src/firefox/src/ui/locales/pt.js @@ -128,6 +128,7 @@ export default { 'sp.slash.show_memory': "Mostrar memória salva do usuário", 'sp.slash.forget_memory': "Esqueça uma memória salva por ID", 'sp.slash.workflows': "Liste e gerencie fluxos de trabalho salvos", + "sp.slash.teach": "Grave suas ações como um fluxo de trabalho salvo", 'sp.slash.run_workflow': "Execute um fluxo de trabalho salvo por ID", 'sp.workflows.run_prompt': "Execute o fluxo de trabalho salvo “{name}”", 'sp.workflows.parameters_for': "Insira os parâmetros para “{name}”. Os valores são usados ​​apenas para esta execução.", @@ -373,6 +374,13 @@ export default { 'sp.workflows.saved': "Fluxo de trabalho salvo “{name}”.", 'sp.workflows.deleted': "Fluxo de trabalho “{name}” excluído.", 'sp.workflows.error': "Erro de fluxo de trabalho: {msg}", + "sp.teach.started": "Ensinando “{name}”. Execute o fluxo e use /teach --end.", + "sp.teach.status": "Ensinando “{name}”: {count} ações capturadas.", + "sp.teach.inactive": "O modo de ensino não está ativo.", + "sp.teach.saved": "Fluxo “{name}” aprendido com {count} etapas.", + "sp.teach.already_active": "O modo de ensino já está ativo nesta aba.", + "sp.teach.agent_running": "Interrompa a execução ativa do agente antes de iniciar o modo de ensino.", + "sp.teach.capture_unavailable": "O modo de ensino não pode capturar esta página. Abra ou recarregue uma página http(s) comum e tente novamente.", 'sp.workflows.reason.no_trace': "Nenhuma execução gravada com sucesso foi encontrada nesta conversa. Ative o rastreamento, conclua a tarefa e salve-a.", 'sp.workflows.reason.no_steps': "Essa execução não possui ações que possam ser salvas com segurança para repetição.", 'sp.workflows.reason.not_found': "Nenhum fluxo de trabalho salvo possui esse ID.", diff --git a/src/firefox/src/ui/locales/ru.js b/src/firefox/src/ui/locales/ru.js index 751383db5..647161709 100644 --- a/src/firefox/src/ui/locales/ru.js +++ b/src/firefox/src/ui/locales/ru.js @@ -770,6 +770,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "Не сохранено: после очистки текст пуст или похож на секрет (пароль, API-ключ, токен).", "sp.memory.reason.not_found": "Сохранённой записи с таким ID нет.", "sp.slash.workflows": "Просмотр и управление сохранёнными сценариями", + "sp.slash.teach": "Записать ваши действия как сохранённый рабочий процесс", "sp.slash.run_workflow": "Запустить сохранённый сценарий по ID", "sp.workflows.run_prompt": "Запустить сохранённый сценарий «{name}»", "sp.workflows.parameters_for": "Введите параметры для «{name}». Значения используются только в этом запуске.", @@ -785,6 +786,13 @@ export default { "sp.workflows.saved": "Сценарий «{name}» сохранён.", "sp.workflows.deleted": "Сценарий «{name}» удалён.", "sp.workflows.error": "Ошибка сценария: {msg}", + "sp.teach.started": "Обучение «{name}». Выполните процесс, затем используйте /teach --end.", + "sp.teach.status": "Обучение «{name}»: записано действий — {count}.", + "sp.teach.inactive": "Режим обучения не активен.", + "sp.teach.saved": "Рабочий процесс «{name}» изучен: {count} шагов.", + "sp.teach.already_active": "Режим обучения уже активен на этой вкладке.", + "sp.teach.agent_running": "Остановите активный запуск агента перед включением режима обучения.", + "sp.teach.capture_unavailable": "Режим обучения не может записывать эту страницу. Откройте или перезагрузите обычную страницу http(s) и повторите попытку.", "sp.workflows.reason.no_trace": "В этом диалоге нет успешного записанного запуска. Включите трассировку, завершите задачу и сохраните её.", "sp.workflows.reason.no_steps": "В этом запуске нет действий, которые можно безопасно сохранить для повтора.", "sp.workflows.reason.not_found": "Сохранённого сценария с таким ID нет.", diff --git a/src/firefox/src/ui/locales/th.js b/src/firefox/src/ui/locales/th.js index 84ed9ab02..14b744f1c 100644 --- a/src/firefox/src/ui/locales/th.js +++ b/src/firefox/src/ui/locales/th.js @@ -770,6 +770,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "ไม่ได้บันทึก: ข้อความว่างหลังการล้างหรือดูเหมือนมีข้อมูลลับ (รหัสผ่าน คีย์ API โทเค็น)", "sp.memory.reason.not_found": "ไม่มีหน่วยความจำที่บันทึกไว้ด้วย ID นี้", "sp.slash.workflows": "แสดงและจัดการเวิร์กโฟลว์ที่บันทึกไว้", + "sp.slash.teach": "บันทึกการทำงานของคุณเป็นเวิร์กโฟลว์ที่บันทึกไว้", "sp.slash.run_workflow": "เรียกใช้เวิร์กโฟลว์ที่บันทึกไว้ด้วย ID", "sp.workflows.run_prompt": "เรียกใช้เวิร์กโฟลว์ที่บันทึกไว้ “{name}”", "sp.workflows.parameters_for": "ป้อนพารามิเตอร์สำหรับ “{name}” ค่าจะใช้สำหรับการเรียกใช้ครั้งนี้เท่านั้น", @@ -785,6 +786,13 @@ export default { "sp.workflows.saved": "บันทึกเวิร์กโฟลว์ “{name}” แล้ว", "sp.workflows.deleted": "ลบเวิร์กโฟลว์ “{name}” แล้ว", "sp.workflows.error": "ข้อผิดพลาดของเวิร์กโฟลว์: {msg}", + "sp.teach.started": "กำลังสอน “{name}” ทำเวิร์กโฟลว์ให้เสร็จ แล้วใช้ /teach --end", + "sp.teach.status": "กำลังสอน “{name}”: บันทึกแล้ว {count} การทำงาน", + "sp.teach.inactive": "โหมดสอนไม่ได้เปิดอยู่", + "sp.teach.saved": "เรียนรู้เวิร์กโฟลว์ “{name}” แล้ว {count} ขั้นตอน", + "sp.teach.already_active": "โหมดสอนเปิดอยู่แล้วในแท็บนี้", + "sp.teach.agent_running": "หยุดการทำงานของเอเจนต์ก่อนเริ่มโหมดสอน", + "sp.teach.capture_unavailable": "โหมดสอนไม่สามารถบันทึกหน้านี้ได้ เปิดหรือโหลดหน้า http(s) ปกติใหม่แล้วลองอีกครั้ง", "sp.workflows.reason.no_trace": "ไม่พบการทำงานที่บันทึกและสำเร็จในการสนทนานี้ เปิดการติดตาม ทำงานให้เสร็จ แล้วบันทึก", "sp.workflows.reason.no_steps": "การทำงานนี้ไม่มีการกระทำที่สามารถบันทึกเพื่อเล่นซ้ำได้อย่างปลอดภัย", "sp.workflows.reason.not_found": "ไม่มีเวิร์กโฟลว์ที่บันทึกไว้ด้วย ID นี้", diff --git a/src/firefox/src/ui/locales/tl.js b/src/firefox/src/ui/locales/tl.js index e29013853..d94f7f8d8 100644 --- a/src/firefox/src/ui/locales/tl.js +++ b/src/firefox/src/ui/locales/tl.js @@ -770,6 +770,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "Hindi na-save: walang laman ang text pagkatapos linisin o mukhang may lihim (password, API key, token).", "sp.memory.reason.not_found": "Walang naka-save na memory na may ganoong ID.", "sp.slash.workflows": "Ilista at pamahalaan ang mga naka-save na workflow", + "sp.slash.teach": "I-record ang iyong mga kilos bilang naka-save na workflow", "sp.slash.run_workflow": "Patakbuhin ang naka-save na workflow ayon sa ID", "sp.workflows.run_prompt": "Patakbuhin ang naka-save na workflow na “{name}”", "sp.workflows.parameters_for": "Ilagay ang mga parameter para sa “{name}”. Para sa run na ito lang gagamitin ang mga value.", @@ -785,6 +786,13 @@ export default { "sp.workflows.saved": "Na-save ang workflow na “{name}”.", "sp.workflows.deleted": "Tinanggal ang workflow na “{name}”.", "sp.workflows.error": "Error sa workflow: {msg}", + "sp.teach.started": "Itinuturo ang “{name}”. Gawin ang workflow, pagkatapos ay gamitin ang /teach --end.", + "sp.teach.status": "Itinuturo ang “{name}”: {count} kilos ang nakuha.", + "sp.teach.inactive": "Hindi aktibo ang teacher mode.", + "sp.teach.saved": "Natutunan ang workflow na “{name}” na may {count} hakbang.", + "sp.teach.already_active": "Aktibo na ang teacher mode sa tab na ito.", + "sp.teach.agent_running": "Itigil ang aktibong agent run bago simulan ang teacher mode.", + "sp.teach.capture_unavailable": "Hindi makuha ng teacher mode ang pahinang ito. Magbukas o mag-reload ng karaniwang http(s) page at subukan muli.", "sp.workflows.reason.no_trace": "Walang matagumpay na naitalang run sa usapang ito. I-on ang tracing, tapusin ang gawain, at i-save ito.", "sp.workflows.reason.no_steps": "Walang aksyon sa run na ito na ligtas i-save para sa muling pagpapatakbo.", "sp.workflows.reason.not_found": "Walang naka-save na workflow na may ganitong ID.", diff --git a/src/firefox/src/ui/locales/tr.js b/src/firefox/src/ui/locales/tr.js index b8d50d954..fa46718bd 100644 --- a/src/firefox/src/ui/locales/tr.js +++ b/src/firefox/src/ui/locales/tr.js @@ -769,6 +769,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "Kaydedilmedi: temizleme sonrasında metin boş kalıyor veya bir gizli bilgi (parola, API anahtarı, token) içeriyor gibi görünüyor.", "sp.memory.reason.not_found": "Bu kimliğe sahip kaydedilmiş bellek yok.", "sp.slash.workflows": "Kayıtlı iş akışlarını listele ve yönet", + "sp.slash.teach": "Eylemlerinizi kayıtlı bir iş akışı olarak kaydedin", "sp.slash.run_workflow": "Kaydedilmiş iş akışını kimliğe göre çalıştır", "sp.workflows.run_prompt": "Kaydedilmiş “{name}” iş akışını çalıştır", "sp.workflows.parameters_for": "“{name}” için parametreleri girin. Değerler yalnızca bu çalıştırmada kullanılır.", @@ -784,6 +785,13 @@ export default { "sp.workflows.saved": "“{name}” iş akışı kaydedildi.", "sp.workflows.deleted": "“{name}” iş akışı silindi.", "sp.workflows.error": "İş akışı hatası: {msg}", + "sp.teach.started": "“{name}” öğretiliyor. İş akışını tamamlayın, ardından /teach --end kullanın.", + "sp.teach.status": "“{name}” öğretiliyor: {count} eylem yakalandı.", + "sp.teach.inactive": "Öğretme modu etkin değil.", + "sp.teach.saved": "“{name}” iş akışı {count} adımla öğrenildi.", + "sp.teach.already_active": "Öğretme modu bu sekmede zaten etkin.", + "sp.teach.agent_running": "Öğretme modunu başlatmadan önce etkin ajan çalışmasını durdurun.", + "sp.teach.capture_unavailable": "Öğretme modu bu sayfayı yakalayamıyor. Normal bir http(s) sayfası açın veya yenileyip tekrar deneyin.", "sp.workflows.reason.no_trace": "Bu konuşmada başarılı ve kaydedilmiş bir çalışma bulunamadı. İzlemeyi açın, görevi tamamlayın ve kaydedin.", "sp.workflows.reason.no_steps": "Bu çalışmada güvenle kaydedilip yeniden oynatılabilecek bir eylem yok.", "sp.workflows.reason.not_found": "Bu kimliğe sahip kayıtlı iş akışı yok.", diff --git a/src/firefox/src/ui/locales/uk.js b/src/firefox/src/ui/locales/uk.js index 55c3fe9b7..a069b61b4 100644 --- a/src/firefox/src/ui/locales/uk.js +++ b/src/firefox/src/ui/locales/uk.js @@ -770,6 +770,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "Не збережено: після очищення текст порожній або схожий на секрет (пароль, ключ API, токен).", "sp.memory.reason.not_found": "Збереженого запису з таким ID немає.", "sp.slash.workflows": "Перегляд і керування збереженими сценаріями", + "sp.slash.teach": "Записати ваші дії як збережений робочий процес", "sp.slash.run_workflow": "Запустити збережений сценарій за ID", "sp.workflows.run_prompt": "Запустити збережений сценарій «{name}»", "sp.workflows.parameters_for": "Введіть параметри для «{name}». Значення використовуються лише для цього запуску.", @@ -785,6 +786,13 @@ export default { "sp.workflows.saved": "Сценарій «{name}» збережено.", "sp.workflows.deleted": "Сценарій «{name}» видалено.", "sp.workflows.error": "Помилка сценарію: {msg}", + "sp.teach.started": "Навчання «{name}». Виконайте процес, потім скористайтеся /teach --end.", + "sp.teach.status": "Навчання «{name}»: записано дій — {count}.", + "sp.teach.inactive": "Режим навчання не активний.", + "sp.teach.saved": "Робочий процес «{name}» вивчено: {count} кроків.", + "sp.teach.already_active": "Режим навчання вже активний у цій вкладці.", + "sp.teach.agent_running": "Зупиніть активний запуск агента перед увімкненням режиму навчання.", + "sp.teach.capture_unavailable": "Режим навчання не може записувати цю сторінку. Відкрийте або перезавантажте звичайну сторінку http(s) і спробуйте ще раз.", "sp.workflows.reason.no_trace": "У цій розмові не знайдено успішного записаного запуску. Увімкніть трасування, завершіть завдання та збережіть його.", "sp.workflows.reason.no_steps": "Цей запуск не містить дій, які можна безпечно зберегти для повтору.", "sp.workflows.reason.not_found": "Немає збереженого сценарію з таким ID.", diff --git a/src/firefox/src/ui/locales/vi.js b/src/firefox/src/ui/locales/vi.js index c965e11dc..9fbe74644 100644 --- a/src/firefox/src/ui/locales/vi.js +++ b/src/firefox/src/ui/locales/vi.js @@ -128,6 +128,7 @@ export default { 'sp.slash.show_memory': "Hiển thị bộ nhớ người dùng đã lưu", 'sp.slash.forget_memory': "Quên bộ nhớ đã lưu bằng ID", 'sp.slash.workflows': "Liệt kê và quản lý quy trình công việc đã lưu", + "sp.slash.teach": "Ghi lại thao tác của bạn thành quy trình đã lưu", 'sp.slash.run_workflow': "Chạy quy trình công việc đã lưu theo ID", 'sp.workflows.run_prompt': "Chạy quy trình làm việc đã lưu “{name}”", 'sp.workflows.parameters_for': "Nhập thông số cho “{name}”. Các giá trị chỉ được sử dụng cho lần chạy này.", @@ -373,6 +374,13 @@ export default { 'sp.workflows.saved': "Quy trình công việc đã lưu “{name}”.", 'sp.workflows.deleted': "Đã xóa quy trình công việc “{name}”.", 'sp.workflows.error': "Lỗi quy trình làm việc: {msg}", + "sp.teach.started": "Đang dạy “{name}”. Hãy thực hiện quy trình rồi dùng /teach --end.", + "sp.teach.status": "Đang dạy “{name}”: đã ghi {count} thao tác.", + "sp.teach.inactive": "Chế độ dạy chưa hoạt động.", + "sp.teach.saved": "Đã học quy trình “{name}” gồm {count} bước.", + "sp.teach.already_active": "Chế độ dạy đã hoạt động trong thẻ này.", + "sp.teach.agent_running": "Hãy dừng lượt chạy tác nhân trước khi bắt đầu chế độ dạy.", + "sp.teach.capture_unavailable": "Chế độ dạy không thể ghi lại trang này. Hãy mở hoặc tải lại một trang http(s) thông thường rồi thử lại.", 'sp.workflows.reason.no_trace': "Không tìm thấy lần chạy nào được ghi thành công trong cuộc trò chuyện này. Kích hoạt tính năng theo dõi, hoàn thành nhiệm vụ rồi lưu lại.", 'sp.workflows.reason.no_steps': "Lần chạy đó không có hành động nào có thể được lưu lại một cách an toàn để phát lại.", 'sp.workflows.reason.not_found': "Không có quy trình làm việc đã lưu nào có ID đó.", diff --git a/src/firefox/src/ui/locales/zh.js b/src/firefox/src/ui/locales/zh.js index 36fad2e11..a11ec7a24 100644 --- a/src/firefox/src/ui/locales/zh.js +++ b/src/firefox/src/ui/locales/zh.js @@ -770,6 +770,7 @@ export default { "sp.memory.reason.invalid_or_sensitive": "未保存：清理后的文本为空，或看起来包含机密信息（密码、API 密钥、令牌）。", "sp.memory.reason.not_found": "没有使用该 ID 的已保存记忆。", "sp.slash.workflows": "列出和管理已保存的工作流", + "sp.slash.teach": "将您的操作记录为已保存的工作流", "sp.slash.run_workflow": "按 ID 运行已保存的工作流", "sp.workflows.run_prompt": "运行已保存的工作流“{name}”", "sp.workflows.parameters_for": "请输入“{name}”的参数。参数值仅用于本次运行。", @@ -785,6 +786,13 @@ export default { "sp.workflows.saved": "已保存工作流“{name}”。", "sp.workflows.deleted": "已删除工作流“{name}”。", "sp.workflows.error": "工作流错误：{msg}", + "sp.teach.started": "正在示教“{name}”。请完成工作流，然后使用 /teach --end。", + "sp.teach.status": "正在示教“{name}”：已捕获 {count} 个操作。", + "sp.teach.inactive": "示教模式未启用。", + "sp.teach.saved": "已学习工作流“{name}”，共 {count} 个步骤。", + "sp.teach.already_active": "此标签页已启用示教模式。", + "sp.teach.agent_running": "请先停止当前 Agent 运行，再启动示教模式。", + "sp.teach.capture_unavailable": "示教模式无法捕获此页面。请打开或重新加载普通 http(s) 页面后重试。", "sp.workflows.reason.no_trace": "当前对话中没有成功且已记录的运行。请启用跟踪、完成任务，然后保存。", "sp.workflows.reason.no_steps": "该运行没有可以安全保存并重放的操作。", "sp.workflows.reason.not_found": "没有使用该 ID 的已保存工作流。", diff --git a/src/firefox/src/ui/sidepanel.js b/src/firefox/src/ui/sidepanel.js index e2ec6b226..3bfab7aef 100644 --- a/src/firefox/src/ui/sidepanel.js +++ b/src/firefox/src/ui/sidepanel.js @@ -507,6 +507,17 @@ const SLASH_COMMANDS = [ { value: '--file', descriptionKey: 'sp.slash.import_config_file', disallowPayload: true, requires: '--import' }, ], }, + { + value: '/teach', + usage: '/teach [--start | --end]', + descriptionKey: 'sp.slash.teach', + action: 'status', + outOfBand: true, + options: [ + { value: '--start', valueLabel: '', descriptionKey: 'sp.slash.teach', action: 'start', takesRemainder: true, outOfBand: true, exclusiveGroup: 'teach-action' }, + { value: '--end', descriptionKey: 'sp.slash.teach', action: 'end', disallowPayload: true, outOfBand: true, exclusiveGroup: 'teach-action' }, + ], + }, { value: '/allow-api', usage: '/allow-api [prompt]', descriptionKey: 'sp.slash.allow_api', action: 'enable', acceptsPayload: true }, { value: '/foreground', usage: '/foreground [prompt]', descriptionKey: 'sp.slash.foreground', action: 'enable', acceptsPayload: true }, { value: '/dangerously-skip-permissions', usage: '/dangerously-skip-permissions [prompt]', descriptionKey: 'sp.slash.dangerously_skip_permissions', action: 'disable', acceptsPayload: true, outOfBand: true }, @@ -3387,6 +3398,60 @@ function bindSavedWorkflowManager(manager) { }); } +function teacherModeFailureMessage(res) { + if (res?.reason === 'already_active') return t('sp.teach.already_active'); + if (res?.reason === 'no_active_session') return t('sp.teach.inactive'); + if (res?.reason === 'agent_running') return t('sp.teach.agent_running'); + if (res?.reason === 'capture_unavailable') return t('sp.teach.capture_unavailable'); + return savedWorkflowFailureMessage(res); +} + +async function showTeacherModeStatus(tabId = currentTabId) { + try { + const res = await sendToBackground('get_teacher_mode', { tabId }); + if (currentTabId !== tabId) return; + if (!res?.ok) throw new Error(res?.error || res?.reason || 'unknown error'); + addPersistentSlashMessage(res.session?.active + ? t('sp.teach.status', { name: res.session.name, count: res.session.actionCount || 0 }) + : t('sp.teach.inactive')); + } catch (error) { + if (currentTabId === tabId) showComposerToast(t('sp.workflows.error', { msg: error.message }), { duration: 7000 }); + } +} + +async function startTeacherMode(name, tabId = currentTabId) { + try { + const res = await sendToBackground('start_teacher_mode', { tabId, name }); + if (currentTabId !== tabId) return; + if (!res?.ok) { + showComposerToast(teacherModeFailureMessage(res), { duration: 7000 }); + return; + } + addPersistentSlashMessage(t('sp.teach.started', { name: res.session?.name || name })); + } catch (error) { + if (currentTabId === tabId) showComposerToast(t('sp.workflows.error', { msg: error.message }), { duration: 7000 }); + } +} + +async function endTeacherMode(tabId = currentTabId) { + try { + const res = await sendToBackground('end_teacher_mode', { tabId }); + if (currentTabId !== tabId) return; + if (!res?.ok) { + showComposerToast(teacherModeFailureMessage(res), { duration: 7000 }); + return; + } + const message = t('sp.teach.saved', { + name: res.workflow?.name || '', + count: res.workflow?.steps?.length || 0, + }); + const warnings = Array.isArray(res.warnings) ? res.warnings.filter(Boolean) : []; + addPersistentSlashMessage(warnings.length ? `${message} ${warnings.join(' ')}` : message); + } catch (error) { + if (currentTabId === tabId) showComposerToast(t('sp.workflows.error', { msg: error.message }), { duration: 7000 }); + } +} + async function showSavedWorkflows(tabId = currentTabId) { try { const res = await sendToBackground('list_saved_workflows'); @@ -6820,6 +6885,21 @@ async function parseSlashCommands(text, tabId = currentTabId, options = {}) { return ''; } + if (command.value === '/teach' && action === 'status') { + await showTeacherModeStatus(tabId); + return ''; + } + + if (command.value === '/teach' && action === 'start') { + await startTeacherMode(payload, tabId); + return ''; + } + + if (command.value === '/teach' && action === 'end') { + await endTeacherMode(tabId); + return ''; + } + if (command.value === '/workflow' && action === 'list') { await showSavedWorkflows(tabId); return ''; diff --git a/test/run.js b/test/run.js index bd96ffcf7..d1f1ef288 100644 --- a/test/run.js +++ b/test/run.js @@ -292,6 +292,12 @@ const SavedWorkflowsCh = await import( const SavedWorkflowsFx = await import( 'file://' + path.join(ROOT, 'src/firefox/src/agent/workflows.js').replace(/\\/g, '/') ); +const TeacherModeCh = await import( + 'file://' + path.join(ROOT, 'src/chrome/src/agent/teacher-mode.js').replace(/\\/g, '/') +); +const TeacherModeFx = await import( + 'file://' + path.join(ROOT, 'src/firefox/src/agent/teacher-mode.js').replace(/\\/g, '/') +); // config-transfer.js is the pure schema/allowlist boundary used by the // /export --config and /import slash-command handlers. @@ -63468,6 +63474,400 @@ test('saved workflow exact-run compiler never substitutes a newer trace', async } }); +test('teacher mode compiles user demonstrations without retaining entered values', () => { + for (const module of [SavedWorkflowsCh, SavedWorkflowsFx]) { + const result = module.compileWorkflowFromDemonstration({ + name: 'Submit account form', + startUrl: 'https://example.com/form?session=start-secret#private', + webbrainVersion: '26.2.2', + skippedActionCount: 2, + actionLimitReached: true, + actions: [ + { + kind: 'field', + pageUrl: 'https://example.com/form?session=field-secret', + value: 'person@example.com', + target: { role: 'textbox', label: 'Email', fieldName: 'email', type: 'email' }, + }, + { + kind: 'field', + pageUrl: 'https://example.com/form', + value: 'correct horse battery staple', + submit: true, + target: { role: 'textbox', label: 'Password', fieldName: 'password', type: 'password' }, + }, + { + kind: 'checked', + pageUrl: 'https://example.com/settings', + checked: true, + target: { role: 'checkbox', name: 'Remember this device' }, + }, + { + kind: 'click', + pageUrl: 'https://example.com/settings', + target: { role: 'button', name: 'Continue', selector: '#unsafe-selector' }, + }, + { + kind: 'navigate', + pageUrl: 'https://example.com/settings', + url: 'https://example.com/complete?token=navigation-secret#done', + }, + ], + }, { now: 1700 }); + + assert.equal(result.reason, ''); + assert.deepEqual(result.workflow.start, { origin: 'https://example.com', pathFamily: '/form' }); + assert.deepEqual(result.workflow.steps.map((step) => step.tool), [ + 'type_ax', 'set_field', 'set_checked', 'click_ax', 'navigate', + ]); + assert.deepEqual(result.workflow.steps[0].args, { + text: { [module.WORKFLOW_PARAM_REF_KEY]: 'email' }, + clear: true, + }); + assert.equal(result.workflow.steps[1].args.submit, true); + assert.equal(result.workflow.parameters[0].sensitive, false); + assert.equal(result.workflow.parameters[1].sensitive, true); + assert.equal(result.workflow.steps[2].expected.value, true); + assert.equal(result.workflow.steps[4].args.url, 'https://example.com/complete'); + assert.deepEqual(result.workflow.stats, { + sourceToolCount: 7, + compiledStepCount: 5, + skippedToolCount: 2, + }); + assert.equal(result.warnings.length, 2); + assert.doesNotMatch( + JSON.stringify(result.workflow), + /person@example\.com|correct horse|start-secret|field-secret|navigation-secret|unsafe-selector|selector/, + ); + } +}); + +test('teacher session store persists only normalized value-free actions', async () => { + for (const module of [TeacherModeCh, TeacherModeFx]) { + const memory = {}; + const storage = { + async get(key) { return { [key]: structuredClone(memory[key]) }; }, + async set(values) { Object.assign(memory, structuredClone(values)); }, + async remove(key) { delete memory[key]; }, + }; + let now = 2000; + const store = module.createTeacherSessionStore(storage, { now: () => now++ }); + const started = await store.start(42, { + name: 'Login flow', + url: 'https://example.com/login?token=start-secret', + webbrainVersion: '26.2.2', + }); + assert.equal(started.changed, true); + assert.equal((await store.start(42, { name: 'Other', url: 'https://example.com/' })).reason, 'already_active'); + + const passwordField = { + kind: 'field', + pageUrl: 'https://example.com/login?token=field-secret', + value: 'do-not-store-me', + target: { role: 'textbox', label: 'Password', fieldName: 'password', type: 'password' }, + }; + const recorded = await store.record(42, passwordField); + assert.equal(recorded.changed, true); + assert.equal(recorded.session.actions.length, 1); + assert.doesNotMatch(JSON.stringify(memory), /do-not-store-me|start-secret|field-secret/); + + const restored = module.createTeacherSessionStore(storage, { now: () => now++ }); + assert.equal((await restored.get(42)).actions.length, 1); + const submitted = await restored.record(42, { ...passwordField, submit: true }); + assert.equal(submitted.session.actions.length, 1, 'confirmed submit must upgrade the field action in place'); + assert.equal(submitted.session.actions[0].submit, true); + await restored.navigation(42, 'https://example.com/account?token=submit-secret'); + assert.equal( + (await restored.get(42)).actions.length, + 1, + 'submit-driven navigation must not add a duplicate navigate step', + ); + assert.equal((await restored.record(42, { + kind: 'click', + pageUrl: 'https://example.com/login', + target: { role: 'button' }, + })).reason, 'unsafe_target'); + const afterUnsafeTarget = await restored.get(42); + assert.equal(afterUnsafeTarget.actions.length, 1); + assert.equal(afterUnsafeTarget.skippedActionCount, 1); + await restored.record(42, { + kind: 'click', + pageUrl: 'https://example.com/login', + target: { role: 'button', name: 'Continue' }, + }); + await restored.navigation(42, 'https://example.com/account?token=click-secret'); + assert.equal((await restored.get(42)).actions.length, 2, 'click-driven navigation must not add a duplicate step'); + await restored.record(42, { + kind: 'click', + pageUrl: 'https://example.com/account', + target: { role: 'button', name: 'Open settings' }, + }); + await restored.navigation(42, 'https://example.com/settings?token=typed-secret', { force: true }); + const navigated = await restored.get(42); + assert.equal(navigated.actions.length, 4, 'typed navigation must remain replayable after a recent click'); + assert.deepEqual(navigated.actions.at(-1), { + kind: 'navigate', + scope: { origin: 'https://example.com', pathFamily: '/account' }, + url: 'https://example.com/settings', + }); + assert.doesNotMatch(JSON.stringify(memory), /submit-secret|click-secret|typed-secret/); + + await restored.start(43, { + name: 'Submit ordering', + url: 'https://example.com/form', + webbrainVersion: '26.2.2', + }); + const orderedField = { + kind: 'field', + pageUrl: 'https://example.com/form', + target: { role: 'textbox', label: 'Search', fieldName: 'q', type: 'search' }, + }; + await restored.record(43, orderedField); + await restored.navigation(43, 'https://example.com/results'); + const lateSubmit = await restored.record(43, { ...orderedField, submit: true }); + assert.equal(lateSubmit.session.actions.length, 1, 'late submit confirmation must remove an observed navigation'); + assert.equal(lateSubmit.session.actions[0].submit, true); + assert.deepEqual(lateSubmit.session.currentScope, { + origin: 'https://example.com', + pathFamily: '/results', + }); + await restored.clear(43); + assert.equal((await restored.clear(42)).changed, true); + assert.equal(await restored.get(42), null); + } +}); + +test('teacher mode rejects every automated run entry and drops agent-owned capture', async () => { + for (const [browser, TeacherMode, AgentClass] of [ + ['chrome', TeacherModeCh, AgentCh], + ['firefox', TeacherModeFx, AgentFx], + ]) { + const memory = {}; + const storage = { + async get(key) { return { [key]: structuredClone(memory[key]) }; }, + async set(values) { Object.assign(memory, structuredClone(values)); }, + async remove(key) { delete memory[key]; }, + }; + const tabId = 2699; + const agent = new AgentClass({}); + const store = TeacherMode.createTeacherSessionStore(storage, { now: () => 5000 }); + const interlock = TeacherMode.createTeacherRunInterlock(store, { + automationOwnsTab: (ownedTabId) => agent.isRunning(ownedTabId), + }); + const guardedKinds = []; + let automatedDispatches = 0; + agent.setRunStartGuard(async (guardedTabId, context) => { + guardedKinds.push(context.kind); + await interlock.guardRunStart(guardedTabId); + }); + agent._processMessageInner = async () => { automatedDispatches += 1; return 'unexpected'; }; + agent._processMessageStreamInner = async () => { automatedDispatches += 1; return 'unexpected'; }; + agent._executeToolBatch = async () => { automatedDispatches += 1; return { action: 'continue' }; }; + + const started = await interlock.start(tabId, { + name: 'Teacher/run exclusion', + url: 'https://example.com/form', + webbrainVersion: 'test', + }); + assert.equal(started.changed, true, `${browser}: teacher mode did not start`); + + const workflow = { + id: 'workflow_teacher_exclusion', + name: 'Blocked workflow', + steps: [{ + id: 'step_1', + tool: 'click_ax', + args: {}, + target: { role: 'button', name: 'Continue' }, + expected: { kind: 'tool_success' }, + }], + }; + const runAttempts = [ + () => agent.processMessage(tabId, 'interactive run', () => {}, 'act'), + () => agent.processMessage(tabId, 'scheduled run', () => {}, 'act', [], { scheduledRun: true }), + () => agent.processMessage(tabId, 'cloud run', () => {}, 'act', [], { cloudRun: true }), + () => agent.replaySavedWorkflow(tabId, workflow), + () => agent.processMessageStream(tabId, 'streaming run', () => {}, 'act'), + ]; + for (const startAttempt of runAttempts) { + await assert.rejects(startAttempt(), (error) => { + assert.equal(error.code, TeacherMode.TEACHER_RUN_CONFLICT_CODE); + return true; + }); + assert.equal(agent.isRunning(tabId), false, `${browser}: rejected run kept the tab claimed`); + } + assert.deepEqual(guardedKinds, ['interactive', 'scheduled', 'cloud', 'workflow', 'interactive']); + assert.equal(automatedDispatches, 0, `${browser}: a rejected run dispatched an automated action`); + + // Defense in depth: even if a trusted content event or navigation arrives + // after an automation path has claimed the tab, none of it may be retained. + agent._runningTabs.add(tabId); + const click = await interlock.record(tabId, { + kind: 'click', + pageUrl: 'https://example.com/form', + target: { role: 'button', name: 'Continue' }, + }); + const type = await interlock.record(tabId, { + kind: 'field', + pageUrl: 'https://example.com/form', + target: { role: 'textbox', name: 'Email', fieldName: 'email', type: 'email' }, + }); + const navigation = await interlock.navigation(tabId, 'https://example.com/next', { force: true }); + agent._runningTabs.delete(tabId); + assert.deepEqual([click.reason, type.reason, navigation.reason], [ + 'agent_running', 'agent_running', 'agent_running', + ]); + assert.equal((await store.get(tabId)).actions.length, 0, `${browser}: agent-owned actions entered teacher history`); + } +}); + +test('teacher capture marks Enter as submit only after an uncancelled form submission', async () => { + const runCapture = async (build, submitState) => { + const listeners = new Map(); + const actions = []; + const form = {}; + class FakeElement { + constructor() { + this.tagName = 'INPUT'; + this.form = form; + this.labels = []; + } + getAttribute(name) { + return { name: 'query', type: 'search', 'aria-label': 'Search' }[name] || ''; + } + matches(selector) { + if (selector === 'input' || selector === 'button,input') return true; + if (selector.includes('textarea') || selector.includes('select') || selector.includes('contenteditable')) return false; + return false; + } + closest(selector) { + if (selector === 'form') return form; + if (selector === 'label') return null; + return this; + } + } + const field = new FakeElement(); + const document = { + activeElement: field, + documentElement: { appendChild() {} }, + addEventListener(type, listener) { + const registered = listeners.get(type) || []; + registered.push(listener); + listeners.set(type, registered); + }, + createElement() { + return { style: {}, setAttribute() {}, remove() {} }; + }, + getElementById() { return null; }, + }; + const window = { + __wb_ax_role: () => 'textbox', + __wb_ax_name: () => 'Search', + }; + window.top = window; + const chrome = { + runtime: { + lastError: null, + onMessage: { addListener() {} }, + sendMessage(message, callback) { + if (message.action === 'get_teacher_mode') callback?.({ session: { active: true, name: 'Search flow' } }); + if (message.action === 'record_teacher_action') actions.push(message.teacherAction); + return Promise.resolve(); + }, + }, + }; + const source = fs.readFileSync( + path.join(ROOT, `src/${build}/src/content/teacher-capture.js`), + 'utf8', + ); + vm.runInNewContext(source, { + chrome, + document, + window, + Element: FakeElement, + location: { href: 'https://example.com/search' }, + URL, + queueMicrotask, + }); + + listeners.get('keydown')[0]({ + isTrusted: true, + key: 'Enter', + repeat: false, + composedPath: () => [field], + }); + assert.equal(actions.length, 1, `${build}: Enter must first record an ordinary field action`); + assert.equal(actions[0].submit, undefined, `${build}: keydown alone must not imply submission`); + + if (submitState) { + listeners.get('submit')[0]({ + isTrusted: true, + target: form, + defaultPrevented: submitState === 'cancelled', + }); + await new Promise((resolve) => setImmediate(resolve)); + } + return actions; + }; + + for (const build of ['chrome', 'firefox']) { + assert.equal((await runCapture(build, null)).length, 1, `${build}: autocomplete Enter gained submit semantics`); + assert.equal((await runCapture(build, 'cancelled')).length, 1, `${build}: cancelled submit was recorded`); + const confirmed = await runCapture(build, 'confirmed'); + assert.equal(confirmed.length, 2, `${build}: confirmed form submission was not correlated`); + assert.equal(confirmed[1].submit, true); + } +}); + +test('teacher slash commands and value-free page capture are wired in both browsers', () => { + for (const build of ['chrome', 'firefox']) { + const panelPath = `src/${build}/src/ui/sidepanel.js`; + const runtime = loadSlashCommandRuntime(panelPath); + assert.equal(runtime.parseSlashInvocation('/teach').action, 'status'); + const start = runtime.parseSlashInvocation('/teach --start Checkout flow'); + assert.equal(start.action, 'start'); + assert.equal(start.payload, 'Checkout flow'); + assert.equal(runtime.slashInvocationIsOutOfBand(start), true); + const end = runtime.parseSlashInvocation('/teach --end'); + assert.equal(end.action, 'end'); + assert.equal(runtime.slashInvocationIsOutOfBand(end), true); + assert.equal(runtime.parseSlashInvocation('/teach --end extra').error, 'invalid-usage'); + + const panel = fs.readFileSync(path.join(ROOT, panelPath), 'utf8'); + assert.match(panel, /sendToBackground\('start_teacher_mode'/); + assert.match(panel, /sendToBackground\('end_teacher_mode'/); + assert.match(panel, /sendToBackground\('get_teacher_mode'/); + + const background = fs.readFileSync(path.join(ROOT, `src/${build}/src/background.js`), 'utf8'); + assert.match(background, /case 'record_teacher_action':/); + assert.match(background, /case 'start_teacher_mode':/); + assert.match(background, /case 'end_teacher_mode':/); + assert.match(background, /agent\.setRunStartGuard\(\(tabId\) => teacherRunInterlock\.guardRunStart\(tabId\)\)/); + assert.match(background, /teacherRunInterlock\.record\(tabId, msg\.teacherAction\)/); + assert.match(background, /teacherRunInterlock\.navigation\(tabId, url, options\)/); + assert.match(background, /scheduler\.isRunning\(tabId\)/); + assert.match(background, /compileWorkflowFromDemonstration/); + + const scheduler = fs.readFileSync(path.join(ROOT, `src/${build}/src/agent/scheduler.js`), 'utf8'); + assert.match(scheduler, /assertRunStartAllowed\?\.\(resolvedTabId, 'scheduled'/); + + const capturePath = `src/${build}/src/content/teacher-capture.js`; + const capture = fs.readFileSync(path.join(ROOT, capturePath), 'utf8'); + assert.match(capture, /event\.isTrusted/); + assert.match(capture, /action: 'record_teacher_action'/); + assert.match(capture, /action === 'flush_teacher_capture'/); + assert.match(capture, /document\.addEventListener\('submit'/); + assert.match(capture, /event\.defaultPrevented/); + assert.match(capture, /sendAction\(\{ \.\.\.pending\.action, submit: true \}\)/); + assert.match(capture, /name: field \? '' : clean\(window\.__wb_ax_name/); + assert.doesNotMatch(capture, /\.value\b/); + const manifest = JSON.parse(fs.readFileSync(path.join(ROOT, `src/${build}/manifest.json`), 'utf8')); + assert.ok(manifest.content_scripts.some((entry) => entry.js?.includes('src/content/teacher-capture.js'))); + } + const cloudRuns = fs.readFileSync(path.join(ROOT, 'src/chrome/src/cloud-runs.js'), 'utf8'); + assert.match(cloudRuns, /startingTabs\.add\(tabId\)[\s\S]*?assertRunStartAllowed\?\.\(tabId, 'cloud'/); +}); + test('saved workflow slash commands are out-of-band and wired in both browsers', () => { for (const panel of ['src/chrome/src/ui/sidepanel.js', 'src/firefox/src/ui/sidepanel.js']) { const runtime = loadSlashCommandRuntime(panel); @@ -63949,6 +64349,117 @@ test('saved workflow clarify telemetry redacts form field values', () => { }); for (const [browser, AgentClass] of [['chrome', AgentCh], ['firefox', AgentFx]]) { + test(`${browser} saved workflow releases its run claim when initialization fails`, async () => { + const workflow = { + id: 'workflow_init_failure', + name: 'Initialization failure', + start: { origin: 'https://example.com', pathFamily: '/form' }, + steps: [{ id: 'step_1', tool: 'navigate', args: { url: 'https://example.com/next' } }], + }; + for (const stage of ['current_url', 'conversation']) { + const tabId = stage === 'current_url' ? 26990 : 26991; + const agent = new AgentClass({ getActive: () => ({ model: 'test-model' }) }); + agent._hydrate = async () => {}; + agent._persist = () => {}; + agent._currentUrl = async () => { + if (stage === 'current_url') throw new Error('current URL initialization failed'); + return 'https://example.com/form'; + }; + agent.ensureConversationId = async () => { + throw new Error('conversation initialization failed'); + }; + + await assert.rejects( + agent.replaySavedWorkflow(tabId, workflow, {}, () => {}, { cloudRun: true }), + /initialization failed/, + ); + assert.equal(agent.isRunning(tabId), false, `${browser}: ${stage} failure leaked the run claim`); + assert.equal( + agent.completionInvariants.has(tabId), + false, + `${browser}: ${stage} failure leaked completion state`, + ); + if (browser === 'chrome') { + assert.equal( + agent._foregroundCaptureTabs.has(tabId), + false, + `${browser}: ${stage} failure leaked capture policy`, + ); + } + } + }); + + test(`${browser} saved workflow retains its run claim until cleanup finishes`, async () => { + const tabId = browser === 'chrome' ? 26992 : 26993; + const workflow = { + id: 'workflow_cleanup_claim', + name: 'Cleanup claim', + start: { origin: 'https://example.com', pathFamily: '/form' }, + steps: [{ + id: 'step_1', + tool: 'navigate', + args: { url: 'https://example.com/next' }, + scope: { origin: 'https://example.com', pathFamily: '/form' }, + expected: { kind: 'url_changed' }, + }], + }; + const agent = new AgentClass({ getActive: () => ({ model: 'test-model' }) }); + let currentUrl = 'https://example.com/form'; + let cleanupStartedResolve; + let releaseCleanupResolve; + const cleanupStarted = new Promise((resolve) => { cleanupStartedResolve = resolve; }); + const releaseCleanup = new Promise((resolve) => { releaseCleanupResolve = resolve; }); + agent._hydrate = async () => {}; + agent._persist = () => {}; + agent.ensureConversationId = async () => 'conversation_cleanup_claim'; + agent._currentUrl = async () => currentUrl; + agent._executeToolBatch = async (_tabId, calls, _messages, onUpdate) => { + const tool = calls[0].function.name; + currentUrl = 'https://example.com/next'; + onUpdate('tool_result', { name: tool, result: { success: true } }); + return { action: 'continue' }; + }; + agent._endSavedWorkflowTraceRun = async () => { + cleanupStartedResolve(); + await releaseCleanup; + }; + + const replay = agent.replaySavedWorkflow(tabId, workflow); + let result; + await cleanupStarted; + try { + assert.equal(agent.isRunning(tabId), true, `${browser}: cleanup released the run claim early`); + await assert.rejects( + agent.replaySavedWorkflow(tabId, workflow), + /already in progress/, + `${browser}: a second run started while cleanup was pending`, + ); + } finally { + releaseCleanupResolve(); + result = await replay; + } + assert.equal(result.status, 'completed'); + assert.equal(agent.isRunning(tabId), false, `${browser}: completed cleanup retained the run claim`); + + currentUrl = 'https://example.com/form'; + agent._endSavedWorkflowTraceRun = async () => { + throw new Error('trace cleanup failed'); + }; + await assert.rejects( + agent.replaySavedWorkflow(tabId, workflow, {}, () => {}, { cloudRun: true }), + /trace cleanup failed/, + ); + assert.equal(agent.isRunning(tabId), false, `${browser}: failed cleanup leaked the run claim`); + assert.equal(agent.completionInvariants.has(tabId), false, `${browser}: failed cleanup leaked completion state`); + if (browser === 'chrome') { + assert.equal( + agent._foregroundCaptureTabs.has(tabId), + false, + `${browser}: failed cleanup leaked capture policy`, + ); + } + }); + test(`${browser} saved workflow replay resolves fresh targets without exposing runtime parameters`, async () => { const workflow = { schema: SavedWorkflowsCh.SAVED_WORKFLOW_SCHEMA,