Advanced Paste additional customizations and PhiSilica provider#46727
Advanced Paste additional customizations and PhiSilica provider#46727khmyznikov wants to merge 49 commits intomainfrom
Conversation
There was a problem hiding this comment.
check-spelling found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
@check-spelling-bot Report🔴 Please reviewSee the 📂 files view, the 📜action log, or 📝 job summary for details.Unrecognized words (10)djwsxzxb These words are not needed and should be removedIPREVIEW ITHUMBNAIL LPCFHOOKPROC LUMA MAXDWORD MRT suntimes timespan VSyncTo accept these unrecognized words as correct and remove the previously acknowledged and now absent words, you could run the following commands... in a clone of the git@github.com:microsoft/PowerToys.git repository curl -s -S -L 'https://raw.githubusercontent.com/check-spelling/check-spelling/c635c2f3f714eec2fcf27b643a1919b9a811ef2e/apply.pl' |
perl - 'https://github.com/microsoft/PowerToys/actions/runs/23875197434/attempts/1' &&
git commit -m 'Update check-spelling metadata'Pattern suggestions ✂️ (1)You could add these patterns to Alternatively, if a pattern suggestion doesn't make sense for this project, add a Errors, Warnings, and Notices ❌ (3)See the 📂 files view, the 📜action log, or 📝 job summary for details.
See ❌ Event descriptions for more information. If the flagged items are 🤯 false positivesIf items relate to a ...
|
There was a problem hiding this comment.
Pull request overview
This PR upgrades the Advanced Paste module to support Phi Silica (Windows AI) by moving Advanced Paste to a self-contained MSIX with package identity, wiring installer + CI/CD signing for that MSIX, and extending Settings/UI + module runtime logic for new AI provider/customization features.
Changes:
- Package Advanced Paste as a self-contained MSIX and integrate deployment/registration via WiX + custom actions and runtime registration fallback.
- Add Phi Silica provider support (provider selection, availability probing from Settings via named pipe, new Fix Spelling & Grammar action with coaching mode).
- Update build/pipeline/signing/versioning files and developer docs to support the new packaging and provider flow.
Reviewed changes
Copilot reviewed 66 out of 66 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/Verification scripts/verify-installation-script.ps1 | Adds an Advanced Paste MSIX presence check (currently not wired into Main execution). |
| src/settings-ui/Settings.UI/ViewModels/AdvancedPasteViewModel.cs | Adds “set default provider” and sync logic for provider active flags; avoids saving UI-only IsActive changes. |
| src/settings-ui/Settings.UI/Strings/en-us/Resources.resw | Adds new Advanced Paste strings (providers/action customization/coaching) and tweaks existing labels. |
| src/settings-ui/Settings.UI/SettingsXAML/Views/AdvancedPastePage.xaml.cs | Adds Phi Silica availability probing via Advanced Paste named pipe; adds provider selection reset and default-provider actions. |
| src/settings-ui/Settings.UI/SettingsXAML/Views/AdvancedPastePage.xaml | Adds Fix Spelling & Grammar settings/coaching UI and provider pickers; adds provider “Default” badge and context menu action. |
| src/settings-ui/Settings.UI/PowerToys.Settings.csproj | Minor project file formatting update. |
| src/settings-ui/Settings.UI.Library/AIServiceTypeRegistry.cs | Registers Phi Silica metadata (icon/name/local model flags). |
| src/settings-ui/Settings.UI.Library/AIServiceTypeExtensions.cs | Adds Phi Silica parsing/normalization and config string mapping. |
| src/settings-ui/Settings.UI.Library/AIServiceType.cs | Adds PhiSilica enum value. |
| src/settings-ui/Settings.UI.Library/AdvancedPasteSettings.cs | Adds a coaching shortcut hotkey accessor (mapping needs update for new action ordering). |
| src/settings-ui/Settings.UI.Library/AdvancedPasteDefaultPrompts.cs | Introduces shared default prompts for built-in actions (Fix Spelling + coaching). |
| src/settings-ui/Settings.UI.Library/AdvancedPasteCustomAction.cs | Adds per-custom-action provider override persistence. |
| src/settings-ui/Settings.UI.Library/AdvancedPasteAdditionalActions.cs | Adds Fix Spelling & Grammar additional action to settings schema/action enumeration. |
| src/settings-ui/Settings.UI.Library/AdvancedPasteAdditionalAction.cs | Extends additional action schema (prompts, coaching prompts, provider overrides, coaching hotkey/enable). |
| src/PhiSilicaLaf.props | Adds MSBuild properties for Phi Silica LAF credentials (currently contains hard-coded values). |
| src/PackageIdentity/BuildSparsePackage.ps1 | Adds unregister/dev-register switches for sparse package script. |
| src/PackageIdentity/AppxManifest.xml | Removes SettingsUI full-trust application entry from sparse package manifest. |
| src/modules/cmdpal/ExtensionTemplate/TemplateCmdPalExtension/Directory.Packages.props | Updates Microsoft.WindowsAppSDK version in template. |
| src/modules/AdvancedPaste/custom.props | Adds XES versioning props for Advanced Paste packaging. |
| src/modules/AdvancedPaste/AdvancedPasteModuleInterface/dllmain.cpp | Adds coaching hotkey registration; adds Fix Spelling action ordering; expands auto-copy-selection behavior. |
| src/modules/AdvancedPaste/AdvancedPasteModuleInterface/AdvancedPasteProcessManager.cpp | Switches to MSIX registration + protocol launch; increases pipe connect timeout; adds direct-exe fallback. |
| src/modules/AdvancedPaste/AdvancedPasteModuleInterface/AdvancedPasteModuleInterface.vcxproj | Removes stray whitespace. |
| src/modules/AdvancedPaste/AdvancedPasteModuleInterface/AdvancedPaste.base.rc | Removes embedded version resource block. |
| src/modules/AdvancedPaste/AdvancedPaste/ViewModels/OptionsViewModel.cs | Adds coaching explanation generation flow + per-format provider overrides; changes settings launch behavior. |
| src/modules/AdvancedPaste/AdvancedPaste/Strings/en-us/Resources.resw | Adds UI strings for Fix Spelling and coaching title. |
| src/modules/AdvancedPaste/AdvancedPaste/Services/PasteFormatExecutor.cs | Adds Fix Spelling execution path and provider/system-prompt overrides. |
| src/modules/AdvancedPaste/AdvancedPaste/Services/KernelServiceBase.cs | Adds provider override parameter to kernel transform API. |
| src/modules/AdvancedPaste/AdvancedPaste/Services/IKernelService.cs | Updates interface to accept provider override. |
| src/modules/AdvancedPaste/AdvancedPaste/Services/IAICredentialsProvider.cs | Adds provider-specific key lookup API. |
| src/modules/AdvancedPaste/AdvancedPaste/Services/EnhancedVaultCredentialsProvider.cs | Implements provider-specific key lookup and fixes PasswordVault retrieval behavior. |
| src/modules/AdvancedPaste/AdvancedPaste/Services/CustomActions/SemanticKernelPasteProvider.cs | Tweaks OpenAI execution settings (ReasoningEffort). |
| src/modules/AdvancedPaste/AdvancedPaste/Services/CustomActions/PhiSilicaPasteProvider.cs | New provider implementation using Microsoft.Windows.AI.Text.LanguageModel and LAF unlock. |
| src/modules/AdvancedPaste/AdvancedPaste/Services/CustomActions/PasteAIProviderFactory.cs | Registers Phi Silica provider. |
| src/modules/AdvancedPaste/AdvancedPaste/Services/CustomActions/ICustomActionTransformService.cs | Extends transform API for system prompt/provider overrides. |
| src/modules/AdvancedPaste/AdvancedPaste/Services/CustomActions/CustomActionTransformService.cs | Selects provider by override ID and allows system prompt overrides; updates API key acquisition to provider-specific keys. |
| src/modules/AdvancedPaste/AdvancedPaste/Services/CustomActionKernelQueryCacheService.cs | Injects localization function for easier testing. |
| src/modules/AdvancedPaste/AdvancedPaste/Program.cs | Changes Main to return an exit code. |
| src/modules/AdvancedPaste/AdvancedPaste/PhiSilicaLafHelper.cs | Adds helper to unlock Phi Silica limited access feature at runtime. |
| src/modules/AdvancedPaste/AdvancedPaste/Package.appxmanifest | Adds MSIX manifest with protocol activation + systemAIModels capability. |
| src/modules/AdvancedPaste/AdvancedPaste/Package-Dev.appxmanifest | Adds dev MSIX manifest variant. |
| src/modules/AdvancedPaste/AdvancedPaste/Models/PasteFormats.cs | Adds Fix Spelling & Grammar action metadata and IPC key. |
| src/modules/AdvancedPaste/AdvancedPaste/Models/PasteFormat.cs | Adds ProviderId on paste formats and passes it into factory methods. |
| src/modules/AdvancedPaste/AdvancedPaste/Helpers/UserSettings.cs | Surfaces Fix Spelling/coaching/provider settings into runtime user settings. |
| src/modules/AdvancedPaste/AdvancedPaste/Helpers/ResourceLoaderInstance.cs | Switches to default ResourceLoader (MSIX resource resolution). |
| src/modules/AdvancedPaste/AdvancedPaste/Helpers/NativeMethods.cs | Removes unused PointInter → System.Windows.Point conversion. |
| src/modules/AdvancedPaste/AdvancedPaste/Helpers/IUserSettings.cs | Adds Fix Spelling/coaching/provider settings to interface. |
| src/modules/AdvancedPaste/AdvancedPaste/AdvancedPasteXAML/Controls/PromptBox.xaml | Adds coaching explanation UI section in preview pane. |
| src/modules/AdvancedPaste/AdvancedPaste/AdvancedPasteXAML/App.xaml.cs | Adds protocol activation parsing; adds named pipe server for Settings Phi status; supports coaching hotkey suffix parsing. |
| src/modules/AdvancedPaste/AdvancedPaste/AdvancedPaste.csproj | Enables MSIX tooling and package generation; adds AI SDK ref; generates LAF credentials source; changes output path and versioning. |
| src/modules/AdvancedPaste/AdvancedPaste.UnitTests/ServicesTests/CustomActionKernelQueryCacheServiceTests.cs | Updates tests for injected localization function. |
| src/modules/AdvancedPaste/AdvancedPaste.UnitTests/Mocks/IntegrationTestUserSettings.cs | Adds new IUserSettings members for tests. |
| src/CmdPalVersion.props | Adds AdvancedPasteVersion property for versioning/installer. |
| installer/PowerToysSetupVNext/Product.wxs | Adds install/unregister custom actions for Advanced Paste MSIX. |
| installer/PowerToysSetupVNext/PowerToysInstallerVNext.wixproj | Passes AdvancedPasteVersion into WiX defines. |
| installer/PowerToysSetupVNext/generateAllFileComponents.ps1 | Stops harvesting Advanced Paste loose assets (now shipped as MSIX). |
| installer/PowerToysSetupVNext/AdvancedPaste.wxs | Installs Advanced Paste MSIX + dependency appx files and cleans install directories on uninstall. |
| installer/PowerToysSetupCustomActionsVNext/CustomAction.def | Exports new Advanced Paste package install/unregister custom actions. |
| installer/PowerToysSetupCustomActionsVNext/CustomAction.cpp | Implements install/unregister custom actions using package registration helpers. |
| doc/devdocs/modules/advancedpaste.md | Documents MSIX identity, debugging flow, installer behavior, and Settings Phi Silica probing. |
| Directory.Packages.props | Updates Windows App SDK package versions (and AI package version). |
| Directory.Build.props | Imports PhiSilicaLaf.props globally. |
| .vscode/launch.json | Updates Advanced Paste executable path for new output structure. |
| .pipelines/v2/templates/job-build-project.yml | Adds versioning step for Advanced Paste and adds MSIX locate/unpack/sign/repack/stage steps. |
| .pipelines/v2/release.yml | Passes Phi Silica LAF properties into release build options. |
| .pipelines/ESRPSigning_core.json | Updates core signing inputs to sign Advanced Paste MSIX output path. |
| .pipelines/ESRPSigning_advancedpaste_msix_content.json | Adds ESRP batch signing policy for Advanced Paste MSIX contents. |
| function Test-AdvancedPastePackage { | ||
| param( | ||
| [string]$InstallPath | ||
| ) | ||
|
|
||
| $advancedPastePath = Join-Path $InstallPath "WinUI3Apps\AdvancedPaste" | ||
| if (Test-Path $advancedPastePath) { | ||
| $msixFiles = Get-ChildItem $advancedPastePath -Filter "*.msix" -ErrorAction SilentlyContinue | ||
| if ($msixFiles) { | ||
| Add-CheckResult -Category "Advanced Paste" -CheckName "AdvancedPaste MSIX Package" -Status 'Pass' -Message "Found $($msixFiles.Count) Advanced Paste MSIX package(s)" | ||
| } | ||
| else { | ||
| Add-CheckResult -Category "Advanced Paste" -CheckName "AdvancedPaste MSIX Package" -Status 'Warning' -Message "No Advanced Paste MSIX packages found" | ||
| } | ||
| } | ||
| else { | ||
| Add-CheckResult -Category "Advanced Paste" -CheckName "AdvancedPaste Module" -Status 'Warning' -Message "Advanced Paste module not found at: $advancedPastePath" | ||
| } | ||
| } |
There was a problem hiding this comment.
Test-AdvancedPastePackage is defined but never invoked from Main (only Test-CommandPalettePackages / Test-ContextMenuPackages are called). This makes the new Advanced Paste verification check dead code; consider calling it alongside CmdPal in both PerUser/PerMachine paths (or removing it if intentionally unused).
| // Coaching shortcut for Fix Spelling and Grammar | ||
| var fixSpellingAction = Properties.AdditionalActions.FixSpellingAndGrammar; | ||
| hotkeyAccessors.Add(new HotkeyAccessor( | ||
| () => fixSpellingAction.CoachingShortcut, | ||
| value => fixSpellingAction.CoachingShortcut = value ?? new HotkeySettings(), | ||
| "FixSpellingAndGrammarCoaching")); | ||
|
|
There was a problem hiding this comment.
GetAllHotkeyAccessors() maps AdvancedPasteAdditionalAction instances to additionalActionHeaderKeys[index], but FixSpellingAndGrammar was added to AdditionalActions.GetAllActions() without adding a corresponding header key. This will shift labels (and could eventually lead to index issues); add a "FixSpellingAndGrammar" entry in additionalActionHeaderKeys in the correct position, or change this logic to avoid relying on positional indexing.
| if (!package::GetRegisteredPackage(packageName, false).has_value()) | ||
| { | ||
| Logger::info(L"AdvancedPaste MSIX not registered. Registering..."); | ||
|
|
||
| const auto installationFolder = get_module_folderpath(); | ||
|
|
||
| #ifdef _DEBUG | ||
| auto msix = package::FindMsixFile(installationFolder + L"\\WinUI3Apps\\AdvancedPaste\\AppPackages\\PowerToys.AdvancedPaste_0.0.1.0_Debug_Test\\", false); | ||
| auto dependencies = package::FindMsixFile(installationFolder + L"\\WinUI3Apps\\AdvancedPaste\\AppPackages\\PowerToys.AdvancedPaste_0.0.1.0_Debug_Test\\Dependencies\\", true); | ||
| #else | ||
| auto msix = package::FindMsixFile(installationFolder + L"\\WinUI3Apps\\AdvancedPaste\\", false); | ||
| auto dependencies = package::FindMsixFile(installationFolder + L"\\WinUI3Apps\\AdvancedPaste\\Dependencies\\", true); | ||
| #endif | ||
|
|
||
| if (!msix.empty()) | ||
| { | ||
| if (!package::RegisterPackage(msix[0], dependencies)) | ||
| { | ||
| Logger::error(L"Failed to register AdvancedPaste MSIX package"); | ||
| } |
There was a problem hiding this comment.
In start_process, if RegisterPackage(msix[0], dependencies) fails you log an error but still proceed to protocol activation. If registration failed (or protocol handler isn’t present yet), the protocol launch will also fail and there’s no fallback. Consider treating registration failure like the "no MSIX found" case (fall back to direct exe launch) or return an error so callers can handle it.
| const auto installationFolder = get_module_folderpath(); | ||
|
|
||
| #ifdef _DEBUG | ||
| auto msix = package::FindMsixFile(installationFolder + L"\\WinUI3Apps\\AdvancedPaste\\AppPackages\\PowerToys.AdvancedPaste_0.0.1.0_Debug_Test\\", false); | ||
| auto dependencies = package::FindMsixFile(installationFolder + L"\\WinUI3Apps\\AdvancedPaste\\AppPackages\\PowerToys.AdvancedPaste_0.0.1.0_Debug_Test\\Dependencies\\", true); | ||
| #else | ||
| auto msix = package::FindMsixFile(installationFolder + L"\\WinUI3Apps\\AdvancedPaste\\", false); | ||
| auto dependencies = package::FindMsixFile(installationFolder + L"\\WinUI3Apps\\AdvancedPaste\\Dependencies\\", true); | ||
| #endif |
There was a problem hiding this comment.
The debug MSIX search path is hard-coded to ...\AppPackages\PowerToys.AdvancedPaste_0.0.1.0_Debug_Test\, but the project’s AppxPackageTestDir is versioned ($(Version)_Test) and doesn’t include Debug in the folder name. If GeneratePackageLocally / CI produces an MSIX, this code likely won’t find it and will never register the package. Prefer searching under ...\WinUI3Apps\AdvancedPaste\AppPackages\ (or the configured test dir) without hard-coding version/config-specific folder names.
| // Check status once (with MSIX identity) and cache | ||
| string status; | ||
| try | ||
| { | ||
| PhiSilicaLafHelper.TryUnlock(); | ||
| var readyState = Microsoft.Windows.AI.Text.LanguageModel.GetReadyState(); | ||
| status = readyState switch | ||
| { | ||
| Microsoft.Windows.AI.AIFeatureReadyState.NotSupportedOnCurrentSystem => "NotSupported", | ||
| Microsoft.Windows.AI.AIFeatureReadyState.NotReady => "NotReady", | ||
| _ => "Available", | ||
| }; | ||
| } | ||
| catch | ||
| { | ||
| status = "NotSupported"; | ||
| } | ||
|
|
||
| Logger.LogDebug($"Phi Silica status: {status}"); | ||
|
|
||
| // Serve status to any client that connects | ||
| while (true) | ||
| { |
There was a problem hiding this comment.
StartPhiSilicaStatusServer caches Phi Silica ready state once at app startup and then serves the same value for the rest of the process lifetime. If the model transitions from NotReady to Ready after download/Windows Update, Settings will continue to report stale status until Advanced Paste is restarted. Consider re-checking LanguageModel.GetReadyState() per connection (or periodically) so the Settings UI reflects the current state.
| Width="36" | ||
| Height="36" | ||
| HorizontalAlignment="Center" /> | ||
| <TextBlock | ||
| HorizontalAlignment="Center" | ||
| Foreground="{ThemeResource TextFillColorSecondaryBrush}" | ||
| Text="Checking Phi Silica availability..." | ||
| TextAlignment="Center" | ||
| TextWrapping="Wrap" /> | ||
| </StackPanel> | ||
| <StackPanel | ||
| x:Name="PhiSilicaAvailablePanel" | ||
| HorizontalAlignment="Center" | ||
| VerticalAlignment="Center" | ||
| Spacing="8" | ||
| Visibility="Collapsed"> | ||
| <FontIcon | ||
| HorizontalAlignment="Center" | ||
| FontFamily="{StaticResource SymbolThemeFontFamily}" | ||
| FontSize="24" | ||
| Foreground="{ThemeResource SystemFillColorSuccessBrush}" | ||
| Glyph="" /> | ||
| <TextBlock | ||
| x:Name="PhiSilicaAvailableText" | ||
| HorizontalAlignment="Center" | ||
| FontWeight="SemiBold" | ||
| TextAlignment="Center" | ||
| TextWrapping="Wrap" /> | ||
| </StackPanel> | ||
| <StackPanel | ||
| x:Name="PhiSilicaNotAvailablePanel" | ||
| Margin="48,0,48,48" | ||
| HorizontalAlignment="Center" | ||
| VerticalAlignment="Center" | ||
| Spacing="8" | ||
| Visibility="Collapsed"> | ||
| <Image Width="36" Source="ms-appx:///Assets/Settings/Icons/Models/WindowsML.svg" /> | ||
| <TextBlock | ||
| x:Name="PhiSilicaNotAvailableTitle" | ||
| HorizontalAlignment="Center" | ||
| FontWeight="SemiBold" | ||
| TextAlignment="Center" | ||
| TextWrapping="Wrap" /> | ||
| <TextBlock | ||
| x:Name="PhiSilicaNotAvailableDescription" | ||
| HorizontalAlignment="Center" | ||
| FontSize="12" | ||
| Foreground="{ThemeResource TextFillColorSecondaryBrush}" | ||
| TextAlignment="Center" | ||
| TextWrapping="Wrap" /> | ||
| <HyperlinkButton | ||
| HorizontalAlignment="Center" | ||
| Content="Learn more about Copilot+ PCs" | ||
| NavigateUri="https://learn.microsoft.com/windows/ai/npu-devices/" /> | ||
| </StackPanel> |
There was a problem hiding this comment.
New Phi Silica UI strings are hard-coded in XAML (e.g., "Checking Phi Silica availability..." and the HyperlinkButton content). This bypasses the app’s localization pattern (x:Uid / .resw) used elsewhere on the page; move these strings to Resources.resw and reference them via x:Uid (or resource bindings).
| if (result == "NotSupported") | ||
| { | ||
| _isPhiSilicaAvailable = false; | ||
| ShowPhiSilicaNotAvailableState( | ||
| "Phi Silica is not available on this device.", | ||
| "A Copilot+ PC with an NPU is required to use Phi Silica. For on-device AI on any Windows PC, consider using Foundry Local."); | ||
| } | ||
| else if (result == "NotReady") | ||
| { | ||
| _isPhiSilicaAvailable = false; | ||
| ShowPhiSilicaNotAvailableState( | ||
| "Phi Silica model is not ready.", | ||
| "The model needs to be downloaded. Check Windows Update for the AI model download progress."); | ||
| } | ||
| else | ||
| { | ||
| _isPhiSilicaAvailable = true; | ||
| ShowPhiSilicaAvailableState("Phi Silica is available and ready on this device."); | ||
| } | ||
| } | ||
| catch (Exception) | ||
| { | ||
| _isPhiSilicaAvailable = false; | ||
| ShowPhiSilicaNotAvailableState( | ||
| "Phi Silica is not available on this device.", | ||
| "Unable to check Phi Silica availability. A Copilot+ PC with an NPU is required."); | ||
| } |
There was a problem hiding this comment.
UpdatePhiSilicaUIAsync uses multiple user-visible strings hard-coded in code-behind (titles/descriptions like "Phi Silica is not available on this device."). These should be moved to localized resources (resw) to match the rest of Settings UI and avoid shipping non-localizable English text.
This pull request introduces significant improvements to the Advanced Paste module’s packaging, signing, and installation process. The most important changes are the migration of Advanced Paste to a self-contained MSIX package (with package identity), robust CI/CD signing and installer integration, and updated documentation and build configurations to support these changes.
Advanced Paste MSIX packaging and installer integration:
Migrated Advanced Paste to a self-contained MSIX package, including its dependencies, and ensured the MSIX is built and output to
WinUI3Apps/AdvancedPaste/. The WiX installer now deploys and registers the MSIX package and its dependencies, with custom actions to install and uninstall the package during setup/uninstall.Added a new ESRP signing policy and CI pipeline steps to locate, unpack, sign, and re-pack the Advanced Paste MSIX and its contents. This ensures the MSIX and its DLLs/EXEs are properly signed before distribution.
Build and configuration updates:
Updated build templates and props to support Advanced Paste versioning and new MSIX output location. This includes adding versioning steps, updating launch configurations, and importing new build properties for Phi Silica support.
Upgraded Windows App SDK NuGet package versions for Advanced Paste and related projects.
Documentation improvements:
These changes together enable Advanced Paste to use Windows AI APIs (Phi Silica) via native package identity, simplify deployment and signing, and improve maintainability and developer experience.