Skip to content

feat: Add test harness support for flag change listeners in Server SDKs#317

Merged
aaron-zeisler merged 7 commits intofeat/fdv2from
aaronz/flag-change-listener-tests
Mar 2, 2026
Merged

feat: Add test harness support for flag change listeners in Server SDKs#317
aaron-zeisler merged 7 commits intofeat/fdv2from
aaronz/flag-change-listener-tests

Conversation

@aaron-zeisler
Copy link
Copy Markdown

@aaron-zeisler aaron-zeisler commented Feb 18, 2026

This pull request is part of a larger set of PRs that implement Flag Change Listener tests for server SDKs. Here's the list of pull requests that I currently have open:


Summary

Flag Change Listener Contract Tests

This PR adds contract tests for the SDK flag change listener feature — the ability for applications to subscribe to notifications when flag configurations or evaluated values change. Listener registration uses dynamic commands (rather than SDK creation params) because listeners are typically added and removed during a client's lifetime, and this approach enables future tests for unregistration and runtime listener management.

Design

The test harness already has a well-established pattern for verifying async, event-driven SDK behavior: the HTTP callback pattern (used by hooks, migrations, etc.). Flag change listeners follow the same approach.

                          ┌─────────────────────────┐
                          │     SDK Test Harness     │
                          │                          │
                          │  1. Create mock stream   │
                          │  2. Start SDK client     │
                          │  3. Register listener ──────────┐
                          │  4. Push flag update     │      │
                          │  5. Assert notification  │      │
                          │         ▲                │      │
                          └─────────│────────────────┘      │
                                    │                       │
                             POST   │                       │  Command
                             notification                   │  (JSON)
                                    │                       │
                          ┌─────────│────────────────┐      │
                          │   Callback Endpoint      │      ▼
                          │   (mock HTTP server)     │ ┌──────────────┐
                          │                          │ │  SDK Test    │
                          │   Receives JSON:         │ │  Service     │
                          │   {                      │ │              │
                          │     "listenerId": "...", │ │  Wraps SDK   │
                          │     "flagKey": "...",    │ │  listener    │
                          │     "oldValue": ...,     │ │  API, POSTs  │
                          │     "newValue": ...      │ │  back on     │
                          │   }                      │ │  change      │
                          └──────────────────────────┘ └──────────────┘

Flow:

  1. The test harness initializes the SDK client with flags loaded via a mock streaming endpoint
  2. The harness sends a register command to the test service, providing a callback URL
  3. The test service calls the SDK's native listener API (e.g., FlagTracker.AddFlagChangeListener in Go)
  4. The harness pushes a flag update through the mock streaming service
  5. The SDK detects the change and fires the listener
  6. The test service POSTs a ListenerNotification to the callback URL
  7. The harness asserts on the notification contents (flag key, old/new values)

Two Listener Types

Listener Type Command Fires when... Notification includes
Flag Change registerFlagChangeListener Flag configuration changes (any update) flagKey only
Flag Value Change registerFlagValueChangeListener Evaluated value changes for a specific context flagKey, oldValue, newValue

The distinction matters: a flag change listener fires on any configuration update (e.g., targeting rule edits), even if the evaluated value stays the same. A flag value change listener only fires when the value the application would see actually differs.

Capabilities

Tests are split across two capabilities so SDKs only run tests for APIs they actually provide:

Capability Required for SDKs
flag-change-listeners Config change listeners All server-side SDKs
flag-value-change-listeners Value change listeners (old/new) Go, Java, .NET, Python, Ruby

The Node.js server SDK, for example, only supports config change events (update / update:KEY) and does not have a native value change listener API, so it advertises only flag-change-listeners.

Test Coverage

Test What it verifies
Flag Change Listener (flag-change-listeners)
Receives notification when flag changes Basic listener fires on update
Fires on config change even when value unchanged Fires regardless of value delta (e.g., targeting rule edit)
Receives notifications for different flags Listener fires for multiple distinct flag updates
Flag Value Change Listener (flag-value-change-listeners)
Receives notification when value changes Reports correct old and new values
Does not notify when value is unchanged Suppresses no-op updates (same value, new version)
Multiple listeners both receive notification Two listeners on the same flag both fire independently
Is context specific Value change is evaluated per-context (user-1 changes, user-2 doesn't)

What SDK Repositories Need to Implement

Each SDK's test service must:

  1. Add capabilities to the GET / status response:
    • "flag-change-listeners" — all SDKs with a config change listener API
    • "flag-value-change-listeners" — only SDKs with a native value change listener API (Go, Java, .NET, Python, Ruby)
  2. Handle new commands on the client command endpoint:
    • registerFlagChangeListener — subscribe to the SDK's flag change API, POST a ListenerNotification to the callback URL when the listener fires
    • registerFlagValueChangeListener (if advertising the capability) — subscribe to the SDK's flag value change API, capturing old/new values and POSTing them to the callback URL
    • unregisterListener — remove a previously registered listener by its ID
  3. Maintain a listener map keyed by listenerId so that unregisterListener can clean up correctly

No changes to the SDK itself are required — only the test service wrapper.


Requirements

  • I have added test coverage for new or changed functionality
  • I have followed the repository's pull request submission guidelines
  • I have validated my changes against all supported platform versions

Related issues

Provide links to any issues in this repository or elsewhere relating to this pull request.

Describe the solution you've provided

Provide a clear and concise description of what you expect to happen.

Describe alternatives you've considered

Provide a clear and concise description of any alternative solutions or features you've considered.

Additional context

Add any other context about the pull request here.


Note

Low Risk
Low risk: mostly adds new contract tests and service-spec/command definitions, and the new listener tests are gated behind opt-in capabilities so existing SDK test services should not be affected unless they advertise them.

Overview
Adds contract-test coverage for flag change listeners in server-side SDK suites, including both general config change listeners and optional value change (old/new) listeners.

Introduces new test-service surface area to support those tests: new capabilities (flag-change-listeners, flag-value-change-listeners), new commands (registerFlagChangeListener, registerFlagValueChangeListener, unregisterListener) and payload types (ListenerNotification), plus a mock HTTP callback endpoint (ListenerCallbackService) and SDKClient helpers to register/unregister listeners. Updates docs/service_spec.md to document the new capabilities.

Written by Cursor Bugbot for commit 88b7c7d. This will update automatically on new commits. Configure here.

Comment thread .cursor/change-listener-tests-research.md Outdated
@aaron-zeisler aaron-zeisler force-pushed the aaronz/flag-change-listener-tests branch 2 times, most recently from 0cd4fa1 to 35a6b80 Compare February 24, 2026 16:56
@aaron-zeisler aaron-zeisler changed the title feat: Add test harness support for Flag Change Listeners feat: Add test harness support for flag change listeners for Server SDKs Feb 24, 2026
@aaron-zeisler aaron-zeisler changed the title feat: Add test harness support for flag change listeners for Server SDKs feat: Add test harness support for flag change listeners in Server SDKs Feb 24, 2026
aaron-zeisler and others added 5 commits February 24, 2026 15:32
Added the "flag-change-listeners" capability constant to service_params.go
and documented it in service_spec.md. This capability indicates that an SDK
supports registering listeners for flag changes and can notify the test
harness via callbacks.

The capability supports testing both general flag change listeners
(notified on any flag configuration change) and flag value change
listeners (notified when a specific flag's evaluated value changes for
a given context).
Define the command constants and parameter structs needed for the
test harness to instruct SDK test services to register and unregister
flag change listeners at runtime.

Three new commands are added to servicedef/command_params.go:

- registerFlagChangeListener: registers a general flag change listener
  that notifies when any flag's configuration changes. Params include
  a listener ID, an optional flag key to filter on, and a callback URI
  where the test service will POST notifications.

- registerFlagValueChangeListener: registers a value-specific listener
  that notifies when a flag's evaluated value changes for a given
  context. Params include a listener ID, flag key, evaluation context,
  default value, and callback URI.

- unregisterListener: removes a previously registered listener by ID.

Unlike existing commands (evaluate, migrate, etc.), these commands are
stateful — the test service must maintain a map of active listeners
between commands. This is the first example of dynamic, post-init
registration in the test service protocol.

Co-authored-by: Cursor <cursoragent@cursor.com>
Define the callback infrastructure needed for flag change listener tests:

- servicedef/command_params.go: add ListenerNotification, the JSON payload
  POSTed by the SDK test service to the callback URI when a listener fires.
  OldValue/NewValue are present only for value change notifications.

- mockld/listener_callback_service.go: add ListenerCallbackService, a mock
  HTTP endpoint that receives POSTed notifications and delivers them to a
  Go channel for consumption by tests. Mirrors HookCallbackService.

- sdktests/testapi_listeners.go: add ListenerCallback, the test API layer
  wrapping ListenerCallbackService with three assertion helpers:
  ExpectFlagChangeNotification, ExpectValueChangeNotification, and
  ExpectNoNotification.

Co-authored-by: Cursor <cursoragent@cursor.com>
Add three methods to SDKClient in sdktests/testapi_sdk_client.go:

- RegisterFlagChangeListener: sends a registerFlagChangeListener command
  to the test service, registering a listener for general flag config
  changes on an optional specific flag key.

- RegisterFlagValueChangeListener: sends a registerFlagValueChangeListener
  command, registering a listener for evaluated value changes for a
  specific flag key and context.

- UnregisterListener: sends an unregisterListener command to remove a
  previously registered listener by ID.

All three follow the existing SDKClient command pattern: a single params
struct argument, SendCommandWithParams, and nil response (registration
and unregistration produce no response body).

Co-authored-by: Cursor <cursoragent@cursor.com>
Expand the listener test suite with additional coverage for both listener
types. For general flag change listeners: verify that notifications fire on
any configuration change (not only value changes), and that registering
with an empty flag key subscribes to changes for all flags. For value
change listeners: verify that multiple independent listeners for the same
flag both receive notifications, and that listener notifications are scoped
to the specific evaluation context they were registered with.

Co-authored-by: Cursor <cursoragent@cursor.com>
@aaron-zeisler aaron-zeisler force-pushed the aaronz/flag-change-listener-tests branch from 35a6b80 to f19469f Compare February 24, 2026 23:32
@aaron-zeisler aaron-zeisler marked this pull request as ready for review February 24, 2026 23:45
@aaron-zeisler aaron-zeisler requested a review from a team as a code owner February 24, 2026 23:45
Comment thread docs/service_spec.md Outdated
Comment thread sdktests/common_tests_listeners.go
Comment thread sdktests/common_tests_listeners.go Outdated
@aaron-zeisler aaron-zeisler force-pushed the aaronz/flag-change-listener-tests branch from 7189b09 to 88b7c7d Compare February 27, 2026 17:40
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

logger.Printf("Could not read body from listener callback.")
w.WriteHeader(http.StatusBadRequest)
return
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Body logged before ReadAll error is checked

Low Severity

The result of io.ReadAll is passed to logger.Printf on line 44 before the err return value is checked on line 45. If the read fails, bytes may be nil or contain only partial data, yet it gets logged as if it were a complete notification. While string(nil) won't panic in Go, this produces misleading debug output. The events_service.go in the same package correctly checks the error before using the data.

Fix in Cursor Fix in Web

@aaron-zeisler aaron-zeisler merged commit 7f958d1 into feat/fdv2 Mar 2, 2026
6 checks passed
@aaron-zeisler aaron-zeisler deleted the aaronz/flag-change-listener-tests branch March 2, 2026 18:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants