Skip to content

opencode: Validate config files using existing schemas - #9692

Open
sestrella wants to merge 2 commits into
nix-community:masterfrom
sestrella:opencode/validate-config
Open

opencode: Validate config files using existing schemas#9692
sestrella wants to merge 2 commits into
nix-community:masterfrom
sestrella:opencode/validate-config

Conversation

@sestrella

@sestrella sestrella commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Description

Make use of schema metadata to validate JSON configuration files.

Checklist

  • Change is backwards compatible.

  • Code formatted with nix fmt or
    nix-shell -A dev --run treefmt.

  • Code tested through nix build .#test-all
    or a targeted nix run .#tests -- <pattern>.

  • Test cases updated/added. See example.

  • Commit messages are formatted like

    {component}: {description}
    
    {long description}
    

    See CONTRIBUTING for more information and recent commit messages for examples.

  • If this PR adds a new module

    • Added myself as module maintainer. See example.
    • Generate a news entry. See News
    • Basic tests added. See Tests
  • If this PR adds an exciting new feature or contains a breaking change.

    • Generate a news entry. See News

@github-actions github-actions Bot added the ai label Jul 21, 2026
@home-manager-ci
home-manager-ci Bot requested a review from delafthi July 21, 2026 06:05
@sestrella
sestrella force-pushed the opencode/validate-config branch 3 times, most recently from db7c5ba to 96ced5c Compare July 22, 2026 01:54

@khaneliman khaneliman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not comfortable merging this in its current form. Beyond the inline activation-ordering issue, resolving mutable remote $schema URLs during activation makes a switch depend on network availability and upstream schema changes. Existing tests do not execute the onChange hook, the repository's own OpenCode configuration fixture fails this validator invocation despite the targeted tests passing.

I'd prefer pinned-schema validation of the generated store artifacts before writeBoundary (ideally at build time), with tests covering valid, invalid, and offline behavior.

Comment thread modules/programs/opencode.nix Outdated
@delafthi

delafthi commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

The schemas are available in opencode.passthru.jsonschema. see

https://github.com/NixOS/nixpkgs/blob/241313f4e8e508cb9b13278c2b0fa25b9ca27163/pkgs/by-name/op/opencode/package.nix#L166-L170

@sestrella
sestrella force-pushed the opencode/validate-config branch from 96ced5c to 13e82cd Compare July 23, 2026 08:26
Comment thread modules/programs/opencode.nix Outdated
let
jsonschemaCli = lib.getExe pkgs.jsonschema-cli;
in
if schema == null then

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a workaround because the theme schema is not exposed by opencode.passthru.jsonschema. For those cases, jsonschema-cli validate performs validations using the $schema metadata from the JSON file.

❯ jsonschema-cli validate ~/.config/opencode/opencode.json
Schema is valid

@delafthi delafthi Jul 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could add that. PR opened -> #544922

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excelent! I'll re-add theme validations then.

@sestrella

sestrella commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

I addressed the concerns. I'll look into the failing checks later this week. I tested locally, and all opencode-activation tests passed. I haven't run the entire opencode test suite, though, so it's possible that the new validations broke existing tests.

❯ nix run .#tests -- opencode-activation
warning: Git tree '/Users/sestrella/code/home-manager' has uncommitted changes
ℹ Discovering tests...
ℹ Running 6 test(s)...

--- Running test 1/6: test-opencode-activation-config-invalid ---
✅ Test passed: test-opencode-activation-config-invalid
ℹ Test directory available at: /nix/store/fsysbqfjkipq58rjblrjndkca8rlvgkc-nmt-test-opencode-activation-config-invalid/tested/

--- Running test 2/6: test-opencode-activation-config-valid ---
✅ Test passed: test-opencode-activation-config-valid
ℹ Test directory available at: /nix/store/l29j28cm80r9fzfd679bqjc8j19f7sqh-nmt-test-opencode-activation-config-valid/tested/

--- Running test 3/6: test-opencode-activation-themes-invalid ---
✅ Test passed: test-opencode-activation-themes-invalid
ℹ Test directory available at: /nix/store/8qxyc2935i4knkiqi9zq98nc31jp8mgx-nmt-test-opencode-activation-themes-invalid/tested/

--- Running test 4/6: test-opencode-activation-themes-valid ---
✅ Test passed: test-opencode-activation-themes-valid
ℹ Test directory available at: /nix/store/3i6v35kqnpjwc9l1mb4czn58mi8c3x08-nmt-test-opencode-activation-themes-valid/tested/

--- Running test 5/6: test-opencode-activation-tui-invalid ---
✅ Test passed: test-opencode-activation-tui-invalid
ℹ Test directory available at: /nix/store/ggm39hqjr8kfclgf4phm9pli1jjicprp-nmt-test-opencode-activation-tui-invalid/tested/

--- Running test 6/6: test-opencode-activation-tui-valid ---
✅ Test passed: test-opencode-activation-tui-valid
ℹ Test directory available at: /nix/store/l6hlqw9401scd5j0v24rdjhd84bjr4jf-nmt-test-opencode-activation-tui-valid/tested/

--- Summary ---
✅ All 6 tests passed!

@sestrella
sestrella force-pushed the opencode/validate-config branch 3 times, most recently from 176ef97 to 54603d3 Compare July 25, 2026 08:19
@sestrella sestrella changed the title opencode: Validate settings via jsonschema-cli opencode: Validate config files using existing schemas Jul 25, 2026
@sestrella
sestrella force-pushed the opencode/validate-config branch from 54603d3 to 257eda2 Compare July 25, 2026 08:57
@sestrella

Copy link
Copy Markdown
Contributor Author

The tests pass locally and on the macos-latest and ubuntu-latest runners; however, these two fail with the following errors:

aarch64-darwin.test-chunk-15
x86_64-linux.test-chunk-19

I need to look into this further.

@sestrella
sestrella force-pushed the opencode/validate-config branch 2 times, most recently from bb87ef5 to 51d601e Compare July 28, 2026 04:10
@sestrella

Copy link
Copy Markdown
Contributor Author

@khaneliman, after some back and forth, I figured out the problem with the tests was caused by test/default.nix mocking the check-jsonschema package only for Linux, which is why I couldn't reproduce it locally. I would appreciate it if you could take a second look.

@delafthi delafthi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sry, for the late review. I really like these changes. Most of the comments are just suggestions.

Comment thread modules/programs/opencode.nix Outdated
${lib.getExe pkgs.check-jsonschema} \
--schemafile ${lib.escapeShellArg schema} \
${lib.escapeShellArgs instances} >&2
'';

@delafthi delafthi Jul 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the incentive behind this option? Is the validator really necessary for this use case?

If we need an escape hatch, I'd prefer and option like:

validate = {
  config = <true|false>;
  tui = <true|false>;
  themes = <true|false>;
};

More broadly, JSON schema validation isn't something unique and other modules could benefit from such a helper that accepts a schema, a list of files to validate and optionally extra arguments. @khaneliman What do you think?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@delafthi I added this option to allow the validator to be customized, as I found a few CLI tools with similar capabilities. That is why in the example field the jsonschema-cli CLI tool is mentioned as an alternative.

More broadly, JSON schema validation isn't something unique and other modules could benefit from such a helper that accepts a schema

I like the idea; ideally, such schemas should be included in the main binary (similar to opencode) to ensure that configuration files are tested against the schema versions provided by the main binary, as those schemas could change over time.

Comment thread modules/programs/opencode.nix Outdated
};
validateOpenCodeThemes = {
schema = "theme";
instances = lib.mapAttrsToList (name: _: "opencode/themes/${name}.json") cfg.themes;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This won't work if cfg.themes is a path to a directory.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@delafthi, good catch! I'll fix it and add more test coverage.

Comment on lines +34 to +37
[[ "$stderr" == *"Schema validation errors"* ]] || {
printf 'Unexpected stderr:\n%s\n' "$stderr"
exit 1
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case you add the functionality to provide additional arguments to check-jsonschema you can add --output-format=json argument to parse the output via jq. This would make it easier to check the output.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds good! I will try that.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@delafthi I added some missing tests to cover the case where cfg.themes is a directory. Additionally, I updated the assertions to use jq to parse the errors.

@sestrella
sestrella force-pushed the opencode/validate-config branch 2 times, most recently from 26265e8 to 61255b0 Compare July 31, 2026 04:58
@sestrella
sestrella force-pushed the opencode/validate-config branch 3 times, most recently from 8932474 to 96cb131 Compare July 31, 2026 06:38
@sestrella
sestrella force-pushed the opencode/validate-config branch from 96cb131 to 4365fef Compare July 31, 2026 06:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants