Automated UI Testing - #666
Merged
Merged
Conversation
Add AutomationProperties.AutomationId and Name to dialogs and controls for enhanced screen reader and UI automation support. Update ShortcutControl and WordBorder to set automation properties programmatically; implement custom AutomationPeer for WordBorder to support Value pattern. Replace Path.GetTempPath with AutomationProfile.GetTemporaryDirectory for temp files. Minor UI tweaks for accessibility and event handling. These changes improve assistive tech usability and test automation readiness.
Added AutomationProperties.AutomationId to nearly all interactive controls across all settings XAML pages to improve accessibility and enable automated UI testing. In select cases, also set AutomationProperties.Name for enhanced context. Updated file/folder pickers and temp file paths to use automation-aware or profile-based directories. No changes to UI logic or behavior; changes are strictly for accessibility, automation, and testability.
Added AutomationProperties.AutomationId (and Name where needed) to windows and controls in EditTextWindow, FirstRunWindow, FullscreenGrab, GrabFrame, QuickSimpleLookup, and SettingsWindow XAML. These changes improve accessibility and enable reliable automated UI testing by providing unique identifiers for screen readers and test frameworks. No behavioral or logic changes; all updates are declarative in XAML.
Centralize system integration and file paths via AutomationProfile. Gate system integration features (context menu, startup, hotkeys, etc.) on profile permissions. Prefer profile-specific directories for temp and output files. Improve hotkey registration error handling. Enhance modularity and testability for automation scenarios.
Introduce AutomationDiagnostics, AutomationProfile, and AutomationSettingsProvider in Text_Grab.Utilities. Enables event logging, automation-aware settings management, and reproducible configuration for CI and automated scenarios. AutomationProfile parses CLI/env for automation mode, manages directories, and supports settings seeding. Diagnostics and settings provider support robust traceability and override default settings when automation is active.
App now detects active AutomationProfile at startup, enabling automation diagnostics, event recording, and disabling system integration features as needed. SettingsService uses profile-specific paths for settings storage and seeding, ensuring isolated automation runs. Added OverrideAiArchCheck setting. Improved argument parsing, exception handling, and minor refactoring for automation scenarios.
Added AutomationProfileTests for TryCreate scenarios including env var and CLI overrides. Added UiAutomationContractTests to enforce AutomationId presence/uniqueness, WordBorder automation peer, and stable automation selectors in XAML and code. Validates source-level and config contracts to prevent UI automation regressions.
This adds interactive-ui.yml, a GitHub Actions workflow for running interactive UI tests on dedicated Windows desktop runners. The workflow supports manual and scheduled triggers, lane selection (smoke, system-integration, mixed-DPI, packaged, ARM64, Copilot+), and options for build type, recording, xUnit test execution, and dry run. It uses a matrix of specialized runners, ensures exclusive desktop access, runs PowerShell scripts for preflight and lane execution, summarizes UI test results, and uploads artifacts per lane.
Added new TextGrab.AutomationHost WPF project targeting .NET 10.0 and Windows 10 SDK 22621.0. Includes App.xaml/cs, MainWindow.xaml/cs for fixture hosting with selectable surfaces (text, multilingual, OCR, QR/barcode, table, empty, contrast, DPI grid), UI for text/metrics/input, and automation instrumentation. Added FixtureOptions for config parsing, FixtureStateWriter for JSON state output, app.manifest for DPI, and project file with test image content. Enables integration and automation testing with detailed state logging and UIA support.
Added TextGrab.SystemIntegrationHelper (.NET 10.0, WPF, unsafe) for system integration tasks. Program.cs provides CLI for mouse/keyboard simulation, clipboard, drag-and-drop, hotkeys, and window queries using user32.dll interop. Includes argument validation, error handling, and result serialization.
Added UiTests/TextGrab.AutomationHost to Text-Grab.sln for UI automation/testing. Registered project with Debug/Release configs for ARM64, x64, x86. No changes to existing projects.
Added comprehensive PowerShell-based UI automation and capability tests for Text Grab. Introduced helper modules for deterministic app launching, fixture management, and capability assertions. Added test scripts for EditText, GrabFrame, QuickLookup, Settings, Lifecycle, PatternsBulk, DisplayHardware, NotificationsTtsShare, OcrCapability, OsArchitecture, PackagedVm, SystemIntegration, Harness, and CliProtocol. Tests cover deterministic UI automation and system integration, with capability-based gating and environment-aware skips. Ensured test isolation, automation health, and resource cleanup.
Comprehensive UI automation and coverage validation for Text Grab: - Added coverage.json mapping 154 checklist items to tests, coverage, and manual exceptions. - Introduced PowerShell modules/scripts for UI automation, preflight checks, test harness, and result reporting. - Added release sign-off and coverage validation scripts with aggregation, enforcement, and Markdown/JSON reporting. - Supported safe, destructive, and capability-gated test lanes (system, VM, DPI, ARM64, Copilot+). - Updated README.md with prerequisites, runner safety, suite structure, and CI integration. - Added helper modules for parsing, summarizing, and enforcing UI test results and sign-off. - All scripts/modules are Windows-only, require .NET 10.0 and WinApp CLI 0.5.x.
.gitignore now excludes UiTests/artifacts/ to prevent tracking UI test artifacts. README.md adds a "UI automation" section, referencing UiTests/README.md for details on UI automation inventory, release sign-off, runner requirements, and opt-in lanes, improving discoverability of UI test infrastructure documentation.
Attach AutomationSettingsProvider to the generated Settings class via a hand-written partial so classic user.config reads/writes land in the active automation profile's isolated directory, deferring to the base LocalFileSettingsProvider when no profile is active. Implement IApplicationSettingsProvider so Upgrade/Reset/GetPreviousVersion are no-ops under a profile, and add a test seam to override the ambient profile plus unit coverage. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a FullscreenGrab suite with a deterministic always-on overlay smoke plus capability-gated real-input tests (toolbar reveal, region drag to clipboard). Add a --move cursor command to SystemIntegrationHelper that double-moves to guarantee a WM_MOUSEMOVE delta so the auto-hiding toolbar reveals itself, wire the suite into the system-integration CI lane and release signoff, and record its coverage entries. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move the lane catalog and selection logic into a select-lanes job that resolves lanes against the event, dispatch input, and repository variables, then emits a dynamic matrix the interactive-ui job consumes. This replaces the per-entry if: filter, which could not reference the matrix context, with a single has_lanes gate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
OverrideCurrentForTests mutates process-global AutomationProfile state. Running in parallel with other classes that touch Settings.Default could redirect a concurrent Save into this test's temporary profile directory. Join the existing non-parallel "Settings isolation" collection. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Added logic to FullscreenGrab to enforce maximized overlay using ShouldForceMaximize, a DispatcherTimer, and LayoutUpdated handler. Cleaned up event/timer on close. Added unit tests for ShouldForceMaximize.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Classification
Introduces a comprehensive UI automation and release sign-off framework for deterministic, capability-gated, and reproducible validation of Text Grab.
PR Summary
This update adds a robust UI automation system with isolated automation profiles, deterministic test orchestration, and coverage mapping, supporting both local and CI/CD validation.
AutomationProfile,AutomationDiagnostics, andAutomationSettingsProviderfor isolated, structured, and profile-scoped automation environments.AutomationIdproperties for all major UI elements.TextGrab.AutomationHostWPF project andTextGrab.SystemIntegrationHelperutility for fixture hosting and real system input simulation.UiTests/README.mdandcoverage.json.