Cannot run `sub.tf.object.jq` after `sub.transform.aggregate.to.array`

[raphael-ru]

Describe the bug

The following does not work when running substation read.

{
    ...
    transforms: [
        sub.tf.aggregate.to.array(settings={ object: { batch_key: 'key' } }),
        sub.tf.object.jq(settings={filter: '.'}),
    ]
}

To Reproduce

Create a in.json with the following input data.

{ "a": "hello", "key": "x"}
{ "b": "world", "key": "x"}
{ "c": "surprise", "key": "y"}

Create a config.jsonnet like this:

local sub = std.extVar('sub');

{
  tests: [
    {
      name: 'aggregate_jq',
      transforms: [
        sub.tf.test.message({ value: { a: 'hello', key: 'x'} }),
        sub.tf.test.message({ value: { b: 'world', key: 'x'} }),
        sub.tf.test.message({ value: { c: 'surprise', key: 'y'} }),
        sub.tf.send.stdout(),
      ],
      condition: sub.cnd.all([
        sub.condition.format.json()
      ]),
    },
  ],
  transforms: [
    sub.tf.aggregate.to.array({ object: { batch_key: 'key' } }),
    sub.tf.object.jq(settings={filter: '.'}),
    sub.tf.send.stdout(),
  ],
}

Then run the following.

substation read config.jsonnet --file in.json

You should see an output similar to this:

Error: transform 123dce80-8942201a: context canceled

Expected behavior

The transform should run successfully.

Build:

• Architecture: MacOS, using substation read .. --file ..
• Version 2.6.0

Additional context

My debugging journey so far:

• The object_jq Transform will get called with a ctx that is already canceled.
• Changing this line to take in a new context will fix the issue. (I'm not sure if this is what we want, though.)
• I tried following the execution flow in cmd/substation/read.go, but I had trouble making sense of the output. I can paste the trace I built here if needed. I would see aggregate get called for each message. Then, at the end there was a failed called to object_jq.
• It seems like substation test passes?

If this should have been a discussion instead, I can move it over!

Thank you for a great library!

[jshlbrd]

You could consider this a bug since similar examples exist throughout the source code, mostly in the Send transforms (look for references to the WithoutCancel method). This is the same root cause as HTTP connections in Go -- when the client disconnects (in this case, the file reader) any operations that are running will be canceled unless the context ignores the cancellation signal.

In theory you would be less likely to see this in a production deployment (since the client is actually remote and not local, like in the substation CLI) but it could still exist there. It could be worth considering to move this cancellation up the entire app stack to the client, but that would change it's behavior (i.e. client requests will execute all transforms regardless of cancellation, like CLI interruption).