Skip to content

Context::map exhausts a multiplexer's channel budget on large workloads #448

Description

@mpzFixApplier

What happens

Context::map gives each item its own child context, and each child context takes a channel from the multiplexer. buffered(concurrency_limit) (added in #403) bounds how many items are in flight, but each item still gets its own channel ID, so a map over N items opens N channels over its lifetime.

For a mux that tracks channels rather than in-flight work, that is still unbounded. A mux frees a channel when its stream is dropped, but the release is handled by the connection task and lags behind the rate at which the sliding window opens new ones so the live-channel count keeps climbing on a large enough workload and eventually hits the mux's cap.

Where we hit it

Driving tlsn (alpha.15) MPC-TLS sessions. Its mux allows 512 concurrent streams (crates/tlsn/src/session.rs), and the item count for the circuit calls scales with the configured max_sent_data. Around a ~7 KB budget, roughly half of all sessions die:

context mux error

That message is the whole diagnostic, which is part of why this took a while to pin down: ContextError's Display prints only its own text, and the source it hides is maximum number of streams reached. Adding {source} to that Display would make this class of failure self-explanatory happy to send it as a separate one-liner if you want it.

Measured on a loopback MPC-TLS run, sweeping max_sent_data with everything else equal:

max_sent_data with the buffered window
~1 KB no failures observed
6144 - 7040 intermittent
7168 7/15 runs ok
8192 and up mostly failing

Small maps are fine, which is why ordinary workloads don't see it — it needs enough items to outrun the mux's channel reclamation.

Suggested fix

Bound the number of channels a map ever opens, not just how many items run at once: distribute items round-robin over at most concurrency_limit lanes, each lane owning one child context and processing its items sequentially. Channel usage becomes min(items.len(), concurrency_limit), concurrency_limit keeps both its meaning and its default, and both parties still derive an identical layout from the item index alone.

With that change the same sweep gives 12/12 at 7168, 12/12 at 8192 and 10/10 at 16384.

I have this implemented and tested against dev (cargo test --workspace green, existing test_map_respects_concurrency_limit still holds since lanes bound concurrency too) and will open a PR referencing this issue. Happy to take it in a different direction if you would rather solve it inside the mux, or by making the channel-id scheme reuse ids across the window.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions