Settings file creation and validation

[liamjpeters]

PR Summary

Extracted and built upon the settings discoverability of my PR #2134 (omitting all of the JSON file format bits and settings refactoring).

• Adds cmdlet New-ScriptAnalyzerSettingsFile, which generates complete settings files with everything set to it's default value.
• Adds cmdlet Test-ScriptAnalyzerSettingsFile, which validates a settings file (All rule names are valid, all settings valid etc).
• Updates cmdlet Get-ScriptAnalyzerRule to include an Options proprety for configurable rules, including default value.
• Fixes ProcessCustomRulePaths Engine Helper to ignore an empty array of CustomRulePaths.

Motivation

It's not easy or convenient to find out what all the rules and rule settings available are. Your options are really to look at this repo, copy a preset/existing settings file, or look through all of the rules pages on MSLearn. Many people do not even know about settings files.

As a module developer, I want a way to discover what the available rules are and what options they have. I want an effortless way to add a settings file to my project that I can configure to my liking.

New-ScriptAnalyzerSettingsFile

By default this creates a ScriptAnalyzer settings file populated with all rule names and rule configuration (set to default values). The file has some comment help for each field.

The file is always named PSScriptAnalyzerSettings.psd1 so that automatic settings discovery in Invoke-ScriptAnalyzer picks it up without any -Settings parameter needed.

Any rule configuration that is an enum - we attempt to list the possible values as a comment after the default value.

Example output of running New-ScriptAnalyzerSettingsFile in this gist

You can optionally base the rule file on a preset. It takes the settings defined in the preset and normalises it all with the comments and all settings fields.

Example output of running New-ScriptAnalyzerSettingsFile -BaseOnPreset CmdletDesign in this gist.

Note: I'm sure some clever person could eventually plumb this into the VSCode extension, as a code action to create a new settings file perhaps.

Test-ScriptAnalyzerSettingsFile

This validates a settings file and tells you exactly what's wrong and where. It checks for:

• Parse errors
• Unknown rule names in IncludeRules, ExcludeRules, and Rules
• Rules in Rules that aren't configurable
• Unrecognised option names
• Option values that don't match the expected type (bool, int, string, arrays)
• Enum string values (e.g. Kind = 'banana' when the valid values are space and tab)
• Invalid severity values

Each problem comes back as a DiagnosticRecord, unless -Quiet mode used - then just a boolean $true`$false` indicating whether the file is valid or not is returned.

I debated for a while whether this should have just been a rule and after some chicken-or-egg thinking, decided it should be a separate cmdlet.

Get-ScriptAnalyzerRule

Added an Options property to the RuleInfo. I have not updated the format file, so it's not shown by default. I didn't know the implications of updating the format file.

Running Get-ScriptAnalyzerRule -Name PSUseConsistentIndentation | select -expand Options gets you:

Name                OptionType                            DefaultValue PossibleValues
----                ----------                            ------------ --------------
Enable              System.Boolean                               False 
IndentationSize     System.Int32                                     4 
Kind                System.String                                space {Space, Tab}
PipelineIndentation System.String  IncreaseIndentationForFirstPipeline {IncreaseIndentationForFirstPipeline, ...

Successful test run in my fork.

PR Checklist

• PR has a meaningful title
  • Use the present tense and imperative mood when describing your changes
• Summarized changes
• Change is not breaking
• Make sure all .cs, .ps1 and .psm1 files have the correct copyright header
• Make sure you've added a new test if existing tests do not effectively test the code changed and/or updated documentation
• This PR is ready to merge and is not Work in Progress.
  • If the PR is work in progress, please add the prefix WIP: to the beginning of the title and remove the prefix when the PR is ready.

[andyleejordan]

Thanks Liam — I reviewed this with help from Claude Opus 4.7: Code quality is high (clean separation of formatting/validation/AST helpers, defensive null checks, proper ShouldProcess on New-ScriptAnalyzerSettingsFile), the new public surface is purely additive (new cmdlets, a new Options property on RuleInfo via a new constructor overload — existing constructors and consumers untouched), and test coverage is genuinely thorough on both happy and error paths.

A few thoughts before I approve:

1. Maintenance cost is real but bounded. Once these ship they're public API; future rule additions will need their configurable options to round-trip cleanly through RuleOptionInfo, and Test-ScriptAnalyzerSettingsFile becomes coupled to the .psd1 settings schema. The enum-discovery heuristic in RuleOptionInfo (the naming-convention match on the option's property type) is clever but silent on mismatch — worth either documenting the convention or considering an explicit attribute, so a future contributor doesn't accidentally ship a rule whose PossibleValues come out empty.
2. Relationship to #2134. You've explicitly carved this off from the JSON-settings refactor, and as far as I can tell #2176 stands alone against the existing .psd1 path. If #2134 lands later, both new cmdlets will need format-aware updates — but that's expected and not a blocker here.
3. vscode-powershell connection. Have you seen vscode-powershell#5385? Users keep getting bitten by the extension's effective defaults differing from Invoke-ScriptAnalyzer's defaults, and the workaround they're given is "author a PSScriptAnalyzerSettings.psd1 by hand," which is exactly the cliff this PR makes shorter. New-ScriptAnalyzerSettingsFile -BaseOnPreset <…> plus Test-ScriptAnalyzerSettingsFile together give them a discoverable starting point and a way to confirm it parses. Worth calling out in the docs/examples?
4. @bergmeister — would value your read on this one before merging, particularly on (1) maintenance posture and on whether the RuleOptionInfo heuristic is something we want to lock in or constrain.

Me: Overally I support merging this. As a new feature I'm less inclined to carefully pick through the whole implementation and will defer to you, especially given the thorough tests, but want to be certain you'll be around to help maintain it if people start using it.

— Drafted by Copilot (Claude Opus 4.7)

[sdwheeler]

Our style guide dictates American spellings.

Suggested change

	customisation.
	customization.

[sdwheeler]

Suggested change

	the **Force** parameter is specified - in which case it is overwritten.
	the **Force** parameter is specified - in which case it's overwritten.

[sdwheeler]

Suggested change

	not from the location of the settings file. This matches `Invoke-ScriptAnalyzer` behaviour.
	Relative paths in `CustomRulePath` are resolved from the caller's current working directory,
	not from the location of the settings file. This matches `Invoke-ScriptAnalyzer` behavior.