feat: add CI-level OpenAPI documentation validation#2112
Open
rainer-friederich wants to merge 4 commits intomainfrom
Open
feat: add CI-level OpenAPI documentation validation#2112rainer-friederich wants to merge 4 commits intomainfrom
rainer-friederich wants to merge 4 commits intomainfrom
Conversation
|
This PR will trigger a minor release when merged. |
Add two validation layers to ensure OpenAPI specs stay in sync with code: Layer 1 (unit tests): Route coverage test that verifies every code route has a corresponding OpenAPI path entry and vice versa. Uses an explicit allowlist for 61 currently undocumented routes with a ratchet mechanism that prevents the count from growing. Layer 2 (IT tests): Response schema validation using AJV that validates actual API response bodies against declared OpenAPI schemas. Added as proof-of-concept to 6 sites endpoint test cases. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
rainer-friederich
force-pushed
the
feat/validate-openapi-schema
branch
from
c542f4d to
a1a32bb
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Extend Layer 2 schema validation with two new capabilities: - Request body validation (expectRequestSchemaValid): validates that request bodies sent in IT tests conform to the declared OpenAPI requestBody schema, proving input types and required fields are documented correctly. - Strict response validation (expectSchemaValidStrict): injects additionalProperties: false into object schemas that don't explicitly set it, catching undocumented fields leaking through DTOs that are missing from the OpenAPI spec. Both are wired into sites IT tests as proof-of-concept. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The @redocly/cli bundle command uses --ext (not --format) to specify the output file extension. The --format flag does not exist and causes CI to fail with "Unknown argument: format". Also writes the bundled spec to a temp file instead of stdout to avoid potential encoding issues, and gitignores the generated artifact. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The bundled spec must be fully dereferenced (no $ref pointers) for AJV to compile schemas. Without --dereferenced, schemas contain references like #/components/schemas/Site that AJV cannot resolve when compiling individual response/request schemas. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
requestBodyschemas, proving input types and required fields are documented correctly.additionalProperties: falseinto object schemas to catch undocumented fields leaking through DTOs that are missing from the spec.npm testandit-postgres) — no workflow changes neededNew assertion helpers for IT tests
expectSchemaValid(res, method, path)expectSchemaValidStrict(res, method, path)expectRequestSchemaValid(body, method, path)Test plan
npm testpasses (7182 tests, including 7 new OpenAPI coverage tests)npm run lintpassesnpx mocha --require test/it/postgres/harness.js --timeout 30000 'test/it/postgres/sites.test.js') validate schema assertionsexpectSchemaValidStrictcatches undocumented response fieldsexpectRequestSchemaValidcatches request body type mismatches🤖 Generated with Claude Code