Skip to content

fix(ci): run influxdb3-core tests when test harness changes#6811

Closed
Copilot wants to merge 3 commits intoclaude/add-codeblock-ci-tests-jRvG9from
copilot/sub-pr-6810
Closed

fix(ci): run influxdb3-core tests when test harness changes#6811
Copilot wants to merge 3 commits intoclaude/add-codeblock-ci-tests-jRvG9from
copilot/sub-pr-6810

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Feb 11, 2026

The workflow triggers on changes to test/**, Dockerfile.pytest, and compose.yaml but only checked for content changes, resulting in test harness modifications being skipped.

Changes:

  • Harness file detection: Check for changes to test/, Dockerfile.pytest, and compose.yaml using exact matches for root-level files
  • Test routing: When only harness files change, run influxdb3-core tests instead of skipping
  • Test command: Added test:codeblocks:influxdb3-core to package.json
  • CI setup: Added mock credentials for content/influxdb3/core/.env.test

Behavior:

# Before: harness changes trigger workflow but skip tests
CHANGED_FILES="" → should-run=false

# After: harness changes run influxdb3-core tests
CHANGED_HARNESS_FILES="test/pytest/conftest.py" → should-run=true, test-products=["influxdb3-core"]

Content changes continue to route to product-specific tests. When both content and harness files change, content-based routing takes precedence.


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 2 commits February 11, 2026 12:25
Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com>
…ency

Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix CI workflow to handle harness file changes fix(ci): run influxdb3-core tests when test harness changes Feb 11, 2026
Copilot AI requested a review from jstirnaman February 11, 2026 12:28
jstirnaman added a commit that referenced this pull request Apr 21, 2026
The workflow triggers on changes to test/**, Dockerfile.pytest, and
compose.yaml, but detect-changes only grepped content/**/*.md. Harness-only
PRs would trigger the workflow and then report 'No content changes'.

Detect harness paths separately. When only harness files change, suggest
the default group (core + telegraf). When both content and harness files
change, merge the default group into the detected product list so that
harness changes are covered.

Supersedes #6811, which carried the same idea against an earlier revision
of this branch.
@jstirnaman
Copy link
Copy Markdown
Contributor

Closing — the harness-detection logic from this PR has been ported directly into #6810 as commit 0c192e8 (the patch here was stale against #6812's rewrite of detect-changes). Thanks @copilot-swe-agent.

@jstirnaman jstirnaman closed this Apr 21, 2026
jstirnaman added a commit that referenced this pull request Apr 22, 2026
* chore(ci): add code-block testing workflow

Add GitHub Actions workflow to run code-block tests on PRs.

Features:
- Automatic detection of changed content files
- Matrix strategy to run product tests in parallel
- 30-minute timeout for test execution
- Smart test selection based on changed files
- Mock credentials for CI environment
- Detailed test summaries and artifact uploads

Also adds `test:codeblocks:parallel` script to package.json
for running tests in parallel locally.

* feat(test): add performance optimization for code block tests

Implement parallel test execution by language and test result caching
to significantly improve test performance.

Features:
- Parallel test execution by language (python, bash, sql)
  - Filter tests by programming language
  - Run language-specific tests independently
  - 59% faster execution with parallel runs

- Test result caching system
  - Hash-based content caching
  - 7-day cache expiration
  - Cache management commands (stats, list, clean, clear)
  - 97% faster on unchanged content (second run)

- New test scripts:
  - test-by-language.sh: Filter and test specific languages
  - cached-test.sh: Cache test results by content hash
  - manage-test-cache.sh: Manage cache entries

- New package.json scripts:
  - test:codeblocks:python/bash/sql: Language-specific tests
  - test:cache:stats/list/clean/clear: Cache management

- Documentation:
  - test/TEST-PERFORMANCE.md: Comprehensive performance guide
  - DOCS-TESTING.md: Added performance optimization section

Performance improvements:
- Sequential: ~45 minutes
- Parallel: ~18 minutes (59% faster)
- Cached (2nd run): ~5 seconds (97% faster)

Analysis:
- 766 testable code blocks (sh: 582, bash: 90, python: 10, sql: 46, shell: 38)
- Language aliases supported (bash→sh/shell, python→py, sql→influxql)
- Smart cache invalidation on content changes

* Update .github/workflows/test.yml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Replace default CI test products with Core and Telegraf (#6812)

* feat(ci): add per-product codeblock testing with default group

- Add support for all products: core, enterprise, v2, v1, telegraf,
  cloud, cloud-dedicated, cloud-serverless, clustered, explorer
- Define default test group (core + telegraf) when no product specified
- Exclude cloud products from automatic CI (manual dispatch only)
- Add placeholder scripts for products without pytest services
- Normalize product name handling (core, influxdb3_core, influxdb3-core)
- Log informative messages when excluded products' content changes

* chore(ci): make codeblock tests manual-only and informational

- Remove pull_request trigger, keep only workflow_dispatch
- Change all exit codes to 0 so workflow never blocks PRs
- Use warnings instead of errors for failed tests
- Simplify job from detect-changes to parse-inputs

* feat(ci): make codeblock tests informational on PRs, manual-only execution

- PRs now trigger detection-only mode that suggests which products to test
- Actual test execution only runs via manual workflow_dispatch
- Add detect-test-products.js script using existing content-utils library
- Properly resolve shared content to affected product pages
- Non-blocking: PRs won't fail due to codeblock test issues

* Update .github/workflows/test.yml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix(ci): add guard for empty products array after parsing

Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com>

* fix(ci): remove redundant output before exit 1

Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com>

---------

Co-authored-by: Jason Stirnaman <jstirnaman@influxdata.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: jstirnaman <212227+jstirnaman@users.noreply.github.com>

* fix(ci): detect harness changes and map to default test group

The workflow triggers on changes to test/**, Dockerfile.pytest, and
compose.yaml, but detect-changes only grepped content/**/*.md. Harness-only
PRs would trigger the workflow and then report 'No content changes'.

Detect harness paths separately. When only harness files change, suggest
the default group (core + telegraf). When both content and harness files
change, merge the default group into the detected product list so that
harness changes are covered.

Supersedes #6811, which carried the same idea against an earlier revision
of this branch.

* feat(ci): start InfluxDB 3 Core with preconfigured token for live tests

When matrix.product == influxdb3_core, generate an offline admin token,
write it to test/.influxdb3/core/.token (the compose secret path) and to
content/influxdb3/core/.env.test, then start the influxdb3-core compose
service with --admin-token-file. The pytest container shares the compose
network and reaches the server by service name (influxdb3-core:8181).

This is the first product in the workflow that actually runs against a
live instance instead of mock credentials. Enterprise can follow the
same pattern once we settle on license handling.

Also capture server logs as an artifact and tear down the stack on job
exit so matrix jobs don't leak state.

* fix(ci): chmod data dirs so influxdb3:3-core container can write

The influxdb:3-core image runs as a non-root user. Host-created
bind-mount dirs are owned by the runner user and not writable by the
container, so the server crashes on startup with:

  failed to persist catalog checkpoint file ... PermissionDenied

Make test/.influxdb3 world-writable before `compose up`.

* fix(ci): make admin token file world-readable for compose secret mount

The influxdb:3-core container runs as a non-root user. When a 0600
host file is bind-mounted through docker compose secrets to
/run/secrets/, the container's user can't read it:

  ERROR Failed to initialize admin token from file:
    Failed to read admin token file: Permission denied

Loosen to 0644. Docs recommend 0600 for production — this is CI with
a throwaway token, not a secrets-management pattern.

* fix(ci): authenticate readiness probe; /ping requires token in Core 3.9+

The previous probe used `curl -fsS /ping` without a token. In Core 3.9
the /ping endpoint requires auth by default (can be disabled with
--disable-authz=ping), so the server returned 401 and curl treated it
as a failure. The server was actually running and responsive — we were
just asking it the wrong way.

Include the admin token in the readiness request. As a bonus, this
verifies the token was loaded correctly before pytest gets its turn.

* chore(ci): make code-block test job truly non-blocking

The workflow comments and the test step's `continue-on-error: true`
both signal that code-block failures are informational, not gating.
But a later step explicitly `exit 1`s when test-status == failed,
which undoes the non-blocking intent and turns the job red.

Replace the hard fail with a `::warning::` annotation so the signal
still surfaces in the PR UI and job summary but doesn't fail the
matrix job. Summary and artifact steps run regardless.

* feat(ci): add influxdb3-enterprise-pytest compose service

Mirrors the influxdb3-core-pytest service but targets
content/influxdb3/enterprise/ and uses an enterprise-specific tmp
volume. Enables yarn test:codeblocks:influxdb3_enterprise to run
against the influxdb3-enterprise server on the compose network.

Also declares named volumes influxdb3-core-tmp and
influxdb3-enterprise-tmp that were previously referenced but not
declared at the top level.

* feat(ci): run InfluxDB 3 Enterprise live tests with preconfigured token

Mirrors the Core startup pattern with three Enterprise-specific pieces:

- Requires INFLUXDB3_ENTERPRISE_LICENSE_EMAIL repo secret. Writes it
  to $HOME/influxdata-docs/.influxdb3/enterprise/.env where the
  compose env_file directive reads it.
- Preconfigured offline admin token at admin-token.json (same schema
  as Core), mounted via the influxdb3-enterprise-admin-token secret.
- Starts the service with `--profile shared` (Enterprise lives in
  that profile, not the default).

Readiness probe uses the admin token against /ping on host port 8181.
Loop extends to 120s since Enterprise activation can take longer than
Core's first-boot catalog creation.

Also replaces the test-case skip for influxdb3_enterprise with the
real yarn script, and adds parallel log-capture / teardown steps so
Enterprise runs don't leak state between matrix jobs.

* chore(ci): run default code-block test group in parallel

test:codeblocks:default ran Core && Telegraf serially, meaning Core
failure skipped Telegraf entirely and the total runtime was sum of
both. Background both, wait on each PID, OR the exit codes so the
overall command fails if either leg fails.

* feat(ci): gate Enterprise pytest on pre-verified license blob; filter suggestions

Enterprise:
  Trial activation requires out-of-band email verification which can't
  complete in headless CI. The previous workflow happily started the
  server and then timed out waiting for a license that would never
  arrive. Gate startup on a new INFLUXDB3_ENTERPRISE_LICENSE_BLOB
  secret (base64-encoded trial_or_home_license file from a manually
  activated instance). When the secret is missing, skip Enterprise
  pytest cleanly with a ::notice:: annotation explaining how to
  enable. When present, decode into
  /var/lib/influxdb3/data/cluster0/trial_or_home_license so the
  server skips activation.

  The Enterprise startup step now carries an id so the downstream
  test case can check the skip-pytest output.

Suggest-tests:
  The PR-mode summary listed v1, explorer, and any other content
  paths as actionable suggestions even though their yarn scripts
  are no-op echoes. Partition the detected products into
  'runnable' (have a compose pytest service) and 'not yet runnable'
  buckets and label them separately so users don't burn time
  running no-ops.

* docs(testing): document CI code-block workflow and Enterprise activation

Add a Code-block-testing-in-CI subsection that explains:
- How pull-request vs workflow_dispatch triggers differ
- Which products run against live servers vs mock credentials
- Where the workflow lives and how to trigger it manually

Add an Enterprise provisioning recipe that walks through:
- Registering a verification email
- Activating Enterprise locally to produce a license file
- Uploading the base64-encoded license as INFLUXDB3_ENTERPRISE_LICENSE_BLOB
- Re-triggering CI to confirm Enterprise starts

Callout flags that trial licenses expire so the blob needs periodic
refresh. Mirrors the inline ::notice:: the workflow surfaces when
the secret is missing.

* feat(ci): include influxdb3_enterprise in default product group

The harness-only and workflow_dispatch fallback paths suggested/ran
only Core + Telegraf. Now that Enterprise is a working product in
CI (skipped cleanly without INFLUXDB3_ENTERPRISE_LICENSE_BLOB, run
live with it), it belongs in the default group — otherwise PRs
that touch harness files never surface Enterprise in Suggest Tests,
and manual dispatches with no products specified skip it.

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants